Performance Test Result Analysis and Test Reports

I like to write this miniature article (4 minutes Read) emphasizing Performance test Result analysis. How important is this activity and guidelines around them! I am hoping this will help budding Performance Professionals to improve their understanding.

Introduction: 

We all know that Performance test analysis is one of the significant steps in the Performance test life cycle and it needs to be given prime importance. This activity is as vital when compared to other steps in the overall Performance test lifecycle. The most important part of Performance test analysis is the interpretation of the test results by the Performance professionals who understand Product & Performance aspects together and then sharing them with concerned stakeholders appropriately.

What to watch out

The following are some of the questions ( Even though there are lot many) that need to be pondered every time a test activity is completed and before preparation of the test Report.

• How good or bad are the test results? Considering both Business and System perspective!
• Can the build/Drop be good enough to be certified for the next stage? What are the reasons? How much confidence you have with the code before deployed to Production?
• What is the takeaway from the test? What data points and decisions that need to be conveyed?
• Can you convert Performance test results into the language your executive/Management team/Product Development/Product Manager understands? This is an art and comes with experience.

Note: IMHO, talking in high fi technical language with too many jargons that don’t convey the intended message and criticality to your audience may become a useless conversation. Talk in their language so they apprehend things better. This is an art that will get developed over some time.

Examples, Talk to Business executive mentioning what will be the additional cost on cloud resources with increased Response time for Transaction x?

Talk to Product owners, Mentioning how the increase in Response time impacts overall customer experience?

• How the test results fair when compared to NFR’s & SLA's defined a while back?
• Are the test results realistic?  

Just sharing the Performance Test Results doesn't mean anything and also not reasonable for someone else to make the business decisions for our Performance work.

As a Performance professional, we should always ask Why/What/When/How/Who and Where to add value addition to the entire activity.

Are there any exceptions?

For a good process-oriented organization there shouldn't be any limitation in following all the steps religiously. But I agree there will be situations and work environments where Performance activity will be treated more like a ritual( than a "Critical step") that helps in No/No Go decisions. In these situations, the onus is on the Performance professionals to provide necessary data and facts that help in better decision making. This is also one of the strong reasons why someone from Performance side should be representing the work in these meetings,

Art of preparing Test Reports.

 Then comes the question how are you preparing the test Reports? These are a few guidelines that will help you to do a good job!

1. The activity of generating the test report should happen in less than a few minutes not hours!
2. The generation of these reports should be automated, and you can use one of the many available scripting languages to get them automated. 
3. It doesn't matter whether the test is successful or not always reports should be created!
4. The test reports should be reviewed, and feedback from Leads/Peers should be incorporated before shared with Stakeholders!
5. Test reports may vary based on the different types of tests that we do. Examples: Individual workload scaling (IWS) tests, Full-fledged Performance tests can have different templates.
6. It’s always good to have a template reviewed and approved before you start using them. I can tell you this will save tons of time

There is a tendency that professionals spend more time creating Reports than spending time analyzing them. This should be Vice versa where more time is spent on analysis.

You should have a checklist handy for the Performance test analysis. This will ensure to get a better outcome!

I will write an article soon on how a good template looks like and the significance of them.

 Stay Blessed!