Testcase Parallelization: Best Practices

Implementing parallel testing, is a topic very widely discussed, so much so that everybody is convinced about its importance. But it is also important to understand that advantages of parallel execution cannot be achieved without a correct strategy. This strategy needs to be decided early on, even before starting automation because, switching tests from serial to parallel execution later is not a simple task. Testing frameworks and tests themselves should be designed to run concurrently without colliding.  In this article, we would discuss about best practices that a team needs to follow, to be able to implement parallel execution of testcases. 

Eliminate testcase Interdependency:

I get asked quite often, how can I execute in parallel, testcases that are interdependent?

The first and foremost rule of parallelization is not having interdependency. A test should not rely on the outcome of another test to run, since parallel tests don't necessarily run in any specific order. That defeats the purpose of parallelization.

Design your framework to cater to parallel execution:

Have a detailed plan ready, before you start automation. Plan out your test steps and your route for executing them. Decide beforehand what data each testcase would need. Data generated should also be independent, which means, no overwrites should be permitted. Use synchronization blocks, catch relevant exceptions at right places in the code, close browser instances when done and initiate a new one for a new testcase. Global / static variables need to be avoided as that can prevent another test running in parallel from accessing it. Thread-safe singletons must be used. Consider all above points while designing a framework.

Avoid long testcases:

Testcases should be short and must not span across more than a couple of screens and taking not more than 1 minute to execute. End to end cases which run for long cannot be candidates for parallel execution.

Maintain testdata externally:

Data should always be maintained outside of testcases, in a csv / Excel / JSON or any preferred way of data maintenance. This makes test suite easy to maintain and testcases self-sufficient.

Efficient clean up strategy:

Whatever data was created during the testruns, should be cleaned as soon as the case is over.

Improve testcase scalability:

When tests are executed on a single machine in parallel threads, their efficiency is dictated by the capacity of the execution machine. Hence, if cost is not a limiting factor, it is always better to have multiple machines execute tests for speed and for scale. Also, having an on-premises infrastructure for parallel execution means that you need a dedicated team to maintain these resources, raising the cost further and diverting focus from automation to resource maintenance. Hence it is always better to opt for leased environment solutions like Browser Stack / SauceLabs, AWS, Azure, etc as they turn out to be cost effective in the longer run.