The Misrepresentation of Automation
During my career, I’ve regularly found myself debunking myths about the automated testing of software. They’re usually the same combination of unfounded fears and unrealistic expectations, and they’re usually very easy to dispel
Automating everything improves efficiency
Of course, automation is known for its ability to add consistency, reusability and repeatability to the tests you run regularly. But if you’re not taking a sensible risk-based approach to coverage, and you’re pumping endless hours into heavily automated checks for edge cases, or the corners of your application that barely get traffic in your regression pack, that is inefficient use of your automation engineer’s time. Automate optimally, not just for coverage. Speaking of which...
Automation will allow us to achieve 100% test coverage
Nothing will ever achieve 100% test coverage on any modern application that operates at anything more than a very basic level. And nor should it - in an agile environment, with a smart risk-based strategy, 100% coverage is both unrealistic and unnecessary.
Automation will make manual testing redundant
While automation is a valuable tool, its main function is the checking of your application’s functions and stability at a basic operating level in a repeatable manner, and it therefore cannot replace a manual tester. Automated checks are best used in conjunction with manual exploratory testing, where automation takes care of the repetitive, mechanical checking to give your manual testers the time and freedom to really get stuck into the application. Plus, automation can only look for what it’s told to look for, whereas a manual tester will observe, investigate and flag anomalies based on their knowledge of the system under test. More drilling down = Less screwing up.
Automation will give instant results
While even a single automated check running correctly is technically a result, to get proper, tangible results and improvements to your throughput could potentially take up to a year. The development and introduction of an automation framework is a project by itself, and its proper introduction and use must be seen as a marathon. Its real value will only come with the achievement of repetition of the created checks over regular periods.
Automation finds more bugs
I accept that automation can uncover more regression issues than manual testing, as it generally has a higher coverage percentage in that area than manual testing, and it of course, it shortens the feedback loop for standardised levels of operation. But there are inherently fewer regression issues in any code base than functional issues introduced with a new feature or a change to an existing one, meaning that the majority of bugs should (and indeed, will) be discovered by manual testing around new functions.
Once the automation framework is complete, the job is done
Unless you’re planning to never touch your application ever again, an automation framework will always require an investment of time for monitoring, maintenance, updating test cases to incorporate new functionality, and removal of redundant tests.
Automation is not a quick-fix or a magic bullet, and it’s not a replacement for manual testing. It’s a time and labour saving mechanism, but it does require an initial investment to set up, and an ongoing commitment to maintenance. It should be checking the basic function and performance of your system in a repeatable and reliable manner so that your manual testers are free to really cut loose and delve deep into the heart (and wander around the edges) of their specialist subject - the system under test.
