Software performance testing

In today's era of digitalization and easier availability of the internet to handheld devices such as mobiles/tablets/wearable smart devices usage of day-to-day applications has gone multifold. People can access these apps from anywhere to everywhere. To top it all in these difficult COVID times, many of us have been forced to be at home and have access to our groceries or any essential/non-essential things just by sitting at home with a click of a button. This has led to unprecedented growth of the user base and volume to several eCommerce/marketplaces and also to the communication apps such as Teams/Zoom etc.

The companies have been working extensively to manage such large volumes so that their consumer/customer experience remains seamless. Software performance testing is one of the most crucial aspects of such a harmonious experience. The software applications are constantly being optimized and fine-tuned based upon the results of the performance test.

Performance Testing is a software testing process used for testing the speed, response time, stability, reliability, scalability, and resource usage of a software application under a particular workload. The main purpose of performance testing is to identify and eliminate the performance bottlenecks in the software application.

The focus of Performance Testing is checking a software program's

• Speed - Determines whether the application responds quickly
• Scalability - Determines the maximum user load the software application can handle.
• Stability - Determines if the application is stable under varying loads

Based on the results of the performance test, the stakeholders can actually identify the key aspect of their application such as various configurations of their hardware(compute, memory, networking, and storage), the number of servers needed to host the application in order to handle the projected volume, spot instances v/s reserved instances in case of cloud-native application.

Begin of the Performance Test - As one starts with the performance test one should ask the following before commencement :

1. Scope of the Test - What all components and subsystems are in the scope of the performance test?
2. What are the projected volume and expected TPS(transactions per second) on the application?
3. How many are the concurrent users of the application?
4. Define the test modeling considering the peak volume versus a sustenance normal load.
5. Segregation of the workload model - In case the application has a multi subsystem, defines the workload for each component.

Types of Tests - There can be several types of performances test to measure the viability of the application/system.

Load Test( Measuring the behavior of the application under a projected load), Stress Test( understand the upper limits capacity of the system), Spike Test(observe the behavior of the system by suddenly increasing or decreasing the load generated by a very large number of users, Soak Test(aka endurance testing, is usually done to determine if the system can sustain the continuous expected load. During soak tests, memory utilization is monitored to detect potential leaks)

Various Parameters Monitored during the coarse of the Performance Test

1. CPU Utilization - The amount of time the processor spends executing non-idle threads.
2. Memory Utilization: the amount of physical memory available to processes on a computer.
3. IOPS: used to benchmark computers storage devices like hard disk drives (HDD), solid-state drives (SSD), and storage area networks (SAN)
4. Disk Time: the amount of time the disk is busy executing a read or write request.
5. Response time - the time from when a user enters a request until the first character of the response is received.
6. Throughput - The rate of a computer or network receives requests per second.
7. Thread counts - An application's health can be measured by the no. of threads that are running and currently active.
8. Database Locks: Identify the several DB deadlocks and bottlenecks, long-running queries, top waits, and index fragmentation.

Performance Testing Process:

Issues faced during the Test

• Long Load time - Load time is normally the initial time it takes an application to start. This should generally be kept to a minimum. While some applications are impossible to make load in under a minute, Load time should be kept under a few seconds if possible.
• Poor response time - Response time is the time it takes from when a user inputs data into the application until the application outputs a response to that input. Generally, this should be very quick. Again if a user has to wait too long, they lose interest.
• Poor scalability - A software product suffers from poor scalability when it cannot handle the expected number of users or when it does not accommodate a wide enough range of users. Load Testing should be done to be certain the application can handle the anticipated number of users.
• Bottlenecking - Bottlenecks are obstructions in a system that degrade overall system performance. Bottlenecking is when either coding errors or hardware issues cause a decrease of throughput under certain loads. Bottlenecking is often caused by one faulty section of code. The key to fixing a bottlenecking issue is to find the section of code that is causing the slow down and try to fix it there. Bottlenecking is generally fixed by either fixing poor running processes or adding additional Hardware. Some common performance bottlenecks are

1. CPU utilization
2. Memory utilization
3. Network utilization
4. Operating System limitations
5. Disk usage

Tools:

There are several open-source and licensed tools available in the market. During the test of one of the applications, I had used the Jmeter(open source) tool along with Time-series Databases(InfluxDB) and Grafana Dashboards for extensive monitoring of the application.

In Software Engineering, Performance testing is necessary before marketing any software product. It ensures customer satisfaction & protects an investor's investment against product failure. Costs of performance testing are usually more than made up for with improved customer satisfaction, loyalty, and retention.