Automation : What, When and What Cannot

Browse the internet , linkedin groups and we can find numerous articles/debates on manual and automation testing, talking about their replacements/who is replacing what. One simple answer available everywhere is “NO” which is same as my answer. In my opinion discussion topic should be what can/cannot we automate AND when should we automate.

What can be automated : From my experience, most eligible candidate for test automation are :

1. First in the list are “Smoke & Sanity Tests”. Such tests should be on top priority to automate since we perform these tests on regular interval at every build release. Build Verification cases also comes into this category
2. Regression tests are another category of tests which can be automated.  
3. Next comes are the functional cases related to different modules of the project. Functional cases involving data volume, data generation are suitable candidate for automation as data generation/testing with data volume and its monitoring is a tedious manual task.
4.  Performance/Load/Stress cases are always better to automate as its almost impossible to generate user loads manually.
5. Unit tests testing the functionality of a piece(s) of code  
6.  System tests covering overall processes can be automated but only when the build or participating modules are in stable state.  

When can be automated : When the project/application under test is somewhat in stable condition, at this point of time we should analyze the product (And different functionality). As an output of this analysis we should come up with necessary smoke & sanity, functional and regression cases/scenarios which can be automated.

Recently in one of my project we started late for R&D so a better approach from my perspective is to deploy some time & resources for automation simultaneously with manual test case creation. This group should start analyzing the requirements and R&D on automation feasibility/script creation/designing framework. One advantage of this is “saving time” since by the time we have bunch of cases to be automated we also have some base ready in the automation. This approach is quite useful when we don't have expertise on automation/ didn’t have dedicated teams for automation, so that by the time we have deliver we are ready with an acceptable level of knowledge base.

Automation for performance cases should start simultaneously with manual test case creation as we have defined performance criteria for the application just after the requirements are freezes.  

Few consideration points for automating tests/what can't be automated -

1. There are certain testing such as “exploratory” , “usability” which can’t be performed using automation tools OR having difficulty to automate.
2. Limitation of project budget AND competency of team with organization are also major factors.
3. Nature of project. Small duration projects are difficult and an overhead to automate.
4.  Projects where there is frequent change of requirement and this affects the base code/nature also are not suitable for automation.