Is automation testing a silver bullet?

In the last few years, there has been a shift in the software industry with regards to testing from manual to automation. Increased levels of automation in QA is one of the key trends highlighted in the World Quality Report. Many organizations are investing a huge amount of money to help testers move from manual to automation, while others are replacing manual testers with automation engineers. And on the far end of the spectrum, there are players who have decided not to have a testing team at all.

As organizations continue to shift gears to implement test automation, the industry is also witnessing a new trend. Many companies are gunning to achieve 100% automation of tests within unrealistic deadlines. This clearly indicates a shift in the focus from quality to the volume (quantity) of tests.

There is no doubt that testing teams in organizations are under immense pressure to meet increasing demands but the core aim is to ensure superior product quality and user experience while ensuring a faster turnaround. And, in order to achieve that we need to optimize testing by integrating automation for the right amount of test cases.

Here are top five actions items or best practices that can help you get that balance right.

#1 Have the right kind of tests at the right place

I am a very big advocate of the test pyramid since the day I learned about it. It requires the team to identify what tests go under which segment of the pyramid. When we try to cover every scenario/test via user interface (UI) automation, this not only increases the maintenance cost but also time to execute tests. When the test pyramid model is not used, then there is a risk of running into duplication of tests. From the unit testing to integration to UI automation, it provides a comprehensive testing base. This approach helps to maximize the value for your automation. It increases coverage, lowers cost and accelerates testing, making it an imperative approach in order to succeed in an agile development environment.

#2 Do not downplay automation effort

One of the major challenges facing automation engineers is unrealistic deadlines set by managers. This often leads to automation engineers writing low-value scripts at a warp speed. And, probably writing more codes than developers at times as they need to test multiple scenarios to validate simple code change. The flip side to this is that they may meet deadlines but not quality parameters per se. Automation shouldn’t be underplayed. Managers need to ideally calibrate with their automation engineers to get a perspective on what should be automated and accordingly set a realistic timeline. This will not only help ensure these tests deliver quality, what they are intended to do but also speed things up in the right way.

#3 Don’t conform to the traditional way of integrating automation, innovate rather

Most of the times, when someone talks about automation, the focus shifts to user interface (UI) automation. UI automation is only one area of automation, based on project need, we should identify other areas as well where tests can be automated. For example, we have had automated tests that would monitor logs in servers and API in production and create an alert if a certain exception occurred. You could choose to automate SMS/Email testing or even validate responsive designs or data. While these may not fit into conventional categories such as UI or API or unit, but, why not have them running if they can add value.

#4 You need not automate everything and that’s ok

Most automation goals are vague in nature such as aiming to achieve ‘100% automation of the tests’. This would force automation engineers to automate for the sake of automating, resulting in a number of flaky tests. Rather, the goal should be to implement effective automated tests with a tighter feedback loop. And here’s what Angie Jones, a Senior Automation Engineer at Twitter, has to say in the same context.

#5 Managers who lead the automation team should be hands-on

Managers who are not from automation background often fail to design effective automation strategy. Sometimes, unit tests are omitted with an excuse of delivery pressure and timelines. The ask for more of UI tests increases to account for the omission, this leads to automation engineers spending more time maintaining tests, leaving less time to write new ones. In that case, time saved in regression testing is spent maintaining automation tests, defeating the very purpose of having automation. Automation engineering is a discipline and therefore it is critical, for whoever is responsible for automation strategy, to have hands-on experience in order to design relevant and effective strategy.

Whether or not automation is a silver bullet, I leave that judgment to my readers, please drop a comment to let me know what you think. Although, what I can most certainly conclude is that automation is a powerful weapon that can kill the beast but it requires skillful, strategic and effective handler.