Continuous Performance Engineering Strategy

Throughout the journey of my career, I have seen and witnessed that software organizations treat the performance test process as a performance engineering process. On the other hand, it is challenging to derive an effective performance engineering process/strategy as well.

This need compelled me to think of a Software Performance Engineering Strategy/Model that can be alluded to as a guide for a company that is establishing a performance engineering process from the beginning (refer to the fig1 diagram below for the summary in an illustration). However, this is not a perfect model, but I believe adopting such a model is the first step to lay the foundation for a sound performance engineering process/culture. Each area of this model has a vast scope to cover, however, I discuss only the most essential basics.

This article is open for suggestions and inputs as it is beneficial for the people who are interested in this subject.

Where to start

Performance engineering is a mindset, culture, and need to be inbuilt to the software development process from the early phases. The rapid development of the technology and customer needs have pushed to acquire performance level feedback early (Shift left performance testing) to avoid identifying performance issues in the latter phases which are costly to fix.

So, these trends enable us to assess performance aspects from the initial phases such as requirement gathering, design and development, etc. Specially diminished the boundaries had, that only QA engineers/testers or developers need to think or make sure the performance quality. So preserving the performance quality has become a collective responsibility/mindset of each stakeholder involved in each phase.  

e.g-   Product Owner, Business Analyst - Target user load that the application needs to cater, what level of end-user page load time that the application should comply with, what user flows need to performance test and their data load requirements, etc.

Development team - How scalable is my product/application, how the data load involved features and read-write operations should be optimally handled considering the production usage patterns etc.

QA Engineers – What performance indicators and thresholds that my application needs to comply with, performance behavior of my application in the production environment, etc. How some performance aspects could cover early along with the functional tests etc.

Non-Functional requirements of the performance test

 In other words, Realistic, performance-centric requirements to model the performance strategy. The scope of the nonfunctional requirements is a very broad subject, so this section only a few basic requirements as follows,

•  Acceptable response time/ Response Time SLAs (Service Level Agreements).
•   Concurrent User Load/ transactions per second (TPS).
• Data Structures, data load involved in a performance workflow/scenario.
• Workflows having high usage and their steps.
• Number of Servers in Production Environment etc. Better to get avg and Maximum values of the above where applicable.

Tips: Nonfunctional requirements can be obtained through Analytics information of the production environment. Analytics information from APM (application performance monitoring) tools, IIS logs, Webserver logs, databases and through other monitoring tools etc.

You will realize how important having NFRs and realistic information to model performance tests and how they act as an input for subsequent stages. So it is a good practice to document NFRs.

One can argue how to distinguish NFRs over functional requirements which is a valid argument and different thing to this scope. I believe that any requirement\input that needs to derive an effective performance test model/strategy is a nonfunctional requirement from the performance engineering perspective.

What covers in the next article?

Realistic Performance Scenario selection

•  Discuss some guidelines and sources that can be referred to identify effective performance test scenarios/workflows.
•  Importance of the NFRs to derive realistic performance scenarios.

Deriving an Effective Performance Test Environment

Discuss how a simple “extrapolation” technique can be utilized to derive a scale down performance test environment by referring to the live/production environment.