What Performance testing?
Continue our topic about the Performance testing, where we answered “Why performance testing?” .... tying a big knot, leaving everyone in confusion of how to fix
Confusion is not bad, but it is the start of discovering new facts that were missed; thats why we need to find some way to measure how good “.... or bad :)” the situation is with your business, product, service or code, each according to his role.
“What level of Performance testing I have already? Is it enough?”
Using simple business terms that everyone can feel, we will identify genuine “Business outcomes” that are expected from each of the levels of “Performance testing” and present in form of :
“8 Check list items”
That will clearly show you “where you at” and “if you need more” .... definitely, you need more! else Performance engineers will lose their jobs :)
▢ I know how much response time single transaction(s) , page call(s) takes.
Few examples we all seen across our career of volunteers running some calls to the system and report the time taken as Performance test results, they trying to help, but ... that may make it worst as may mislead decision makers giving them false confidence .... Or half chance for those whom want tick the performance testing check box :P
The most i hate, that some name those single calls as "Performance test" and refer to the rest as "Load test, etc." .... Man, its like defining a space ship as a vehicle!
✪ Performance talented
➼ Goal achieved “Nothing ...yet”
▢ I know the average response time transaction(s) takes, under realistic production load.
Thats the proper way to do it! calls to the system must simulate as accurately as possible the amount and type of calls to the system (preferred during peak time), and the response time must be based on either average or percentiles .... average is good and easier tho :)
✪✪ Performance scripting associate
➼ Goal achieved “Minimum validation, Scripting skills to employ”
▢ I know how long my system can operates under realistic load without showing errors, what errors will show first and how can avoid.
Knowing well the expected bottlenecks to be hit first and mapping the cause to run duration, resource utilisation (cpu, memory, network, disk-space, etc ...) is a great success, and allows setting proper operational plan .... Discover that before the customer does, at least plan better the Ops team vacations :)
▢ I know the maximum users capacity that can use my system.
System breaks at launching day is not cool! we must make sure that the system can handle the amount of calls by customer and try this before, “guess what” everyone wants to see the new function deployed last night .... here we goes :)
✪✪✪ Performance Testing
➼ Goal achieved “What can be measured can be controlled”
Fairly talking, achieving here makes acceptable “Performance testing” coverage where risks are controlled; but still knowing exact machine resources usage figures and doing proper root causes analysis is a great asset and raise the quality of “Performance testing" applied.
⚠ So far, you are good to control your "Current" product, for "Now" ... leaving Future for the unknown
▢ I know until when in future the system will preserve same performance without getting affected by increasing data volumes, and have idea of how to deal with such.
“For now” ? true, because as a normal result of running the system for long time data volumes shall increase in databases, search engines and some of the listing pages/transactions; so unless the effect of volume increase on performance (trend) well studied, things will get out of control sooner or later.
▢ I know the maximum users capacity whom can use the system within accepted performance, and i know how “Exactly” system resources can be scaled up to cover expected new customers for next 1 year (at least)
“Current product”? also true, because unless you know your system limits and how can be scaled up to cover specific number of new users, better stop selling it .... sadly
▢ I know well that system environment tuned in optimised way for “best performance” and/or “minimum cost”
Tuning is to be in ultimate control of the resources the system needs and assign safely what's needed, you can be either focused on minimum resources to cut costs or best fit resources for best response time, if lucky enough both matches .... ☘ happy days!
✪✪✪✪ Performance Engineering
➼ Goal achieved “What can be modelled can be extrapolated”
▢ I can discover any probable performance risk within few hours of introducing the new code/configuration
The early to discover the issue, the cheaper you fix specially when it comes to performance which usually done on staging as last check, imagine revisiting back the code after all boxes checked, repeat the cycle again .... embarrassing really, how to tell that to PM :D
But setting such shift left Performance coverage, which probably will be handled during nightly builds or earlier as unit Performance tests, that requires rare skills ( me, maybe :P ), It is not common approach in known classic performance testing but a promising trend as have great values in return.
✪✪✪✪✪ Performance Architecture
➼ Goal achieved “Full CI/CD Performance validation”
______________________________________________________________________
Summary check list:
▢ I know how much response time single transaction(s) , page call(s) takes.
▢ I know the average response time transaction(s) takes, under realistic production load.
▢ I know how long my system can operates under realistic load without showing errors, what errors will show first and how can avoid.
▢ I know the maximum users capacity that can use my system.
▢ I know until when in future the system will preserve same performance without getting affected by increasing data volumes, and have idea of how to deal with such.
▢ I know the maximum users capacity whom can use the system within accepted performance, and i know how “Exactly” system resources can be scaled up to cover expected new customers for next 1 year (at least)
▢ I know well that system environment tuned in optimised way for “best performance” and/or “minimum cost”
▢ I can discover any probable performance risk within few hours of introducing the new code/configuration
_______________________________________________________________
In fact, It is not the easy to assure an answer that suits all “Business scale” nor “Business domains”, so whats applies to bank and financial transactions for example don't apply to news website; But in general keeping aware of your Business whilst going through the checklist gives you your own answers.
I hope i did well in simplifying complex technical domain into organised form for everyone to understand, regardless of technical concerns on naming or ordering or using non-professional terms that may bring more confusion to non-technical audience.
Please know all pure from my thoughts and nothing copied, looking forward for sharing more .... Cheers ;)