Test Pyramid as a Risk Filter

Based on Mike Cohn’s, there are so many forms and interpretations of the Test Automation Pyramid. All point out the same thing it should have more number of unit tests, followed by service (consisting of API, Integration, Component) tests and finally UI tests.

There’s another way of looking at Test Pyramid as a Bug-Filter.

But none mention non-functional testing. Does that mean non-functional testing is not automated or part of test automation? For me test automation stands for – tests that are automated, run by a tool by executing test-scripts, and asserted pass or fail depending on our expectations.

At least it should say Functional Test Automation Pyramid. But I can see functional test automation is the most widely implemented form of automation and it has kind of become synonymous with test automation.

Also, the Manual & Exploratory Testing bit is not captured properly. It has just been added at the end. Exploratory Testing can happen at any level.

I will now explain how I view this test pyramid or rather a triangle. So here is my interpretation.

Software Testing is the process of identifying RISKS and not about finding BUGS. Based on this, Imagine the test pyramid to be upside down as a risk filter identifying risks for the application.  

Risk Filter consists of Functional Testing, Security Testing, Performance Testing, Other types of testing (Visual UI, Accessibility, A/B, Usability, and many more), and Manual/Exploratory testing. These are the balanced test model to achieve quality. 

Functional Testing consists of Unit, Service (Component, Integration, API – Application Programming Interface), and UI (User Interface) testing. Another testing can be Visual UI, Accessibility, A/B testing, usability testing, and many other types of testing.

A newly developed application or feature enters into the risk filter with the purpose of identifying risks – both Functional and Non-functional, meaning carrying out functional tests in UI, Service, and Unit level and identifying functional risks. Similarly Performance testing for identifying Performance risks, Security testing for Security risks, likewise Other types of testing identifying Other risks. So all these risks identified are by automation – asserting our expectations or checking the KNOWN behaviors. These are our known or identified risks.

What about Unknowns/new information? These will be identified by Exploratory Testing, so all that remains in the filter or pyramid is Manual/Exploratory testing identifying unknown risks in all the areas of testing, for e.g. in functional (including Unit, API, UI), Security, Performance, and Others. Manual/Exploratory testing can be done at each and every level and also in various types of testing. Say for example at unit-level trying out negative test-data, at API level too we can do exploratory testing, similarly in case of security testing writing SQL queries to identify vulnerabilities, likewise in case of accessibility testing manually checking the correct tab-flow, etc. and so on.

Do we need to fix all of the identified risks? Absolutely NOT. Team (Developer/QA/BA) and Product Owner/Stakeholder will prioritize it on a scale of say high, medium, low, etc., and fixed as per the priority.

Once all the prioritized risks are fixed then the Stakeholder/Product Owner can release the application to production with KNOWN risks.