The software development lifecycle (SDLC) is a structured process that guides development teams in creating excellent-quality software successfully and cost-effectively.
Stages of
Load
Testing
Stages of Load Testing
Facebook-f
Linkedin-in
X-twitter
Link Copied
- February 12, 2024
This article outlines the significance and advantages of load testing, providing valuable insights for application owners and guiding readers through the primary stages of the process. The necessity of load testing for web services or mobile applications is highlighted by the potential loss of new business due to sluggish system response times. In a fiercely competitive web services industry, such outcomes are unfortunately common. Consider a scenario where a recently launched pizza ordering website, operating smoothly during regular business hours, faces a surge in traffic leading up to Pizza Day. Despite the perfect timing for promotion, the site experiences sudden halts as new commands flood in, resulting in slow response times and frozen orders mid-checkout. Confusion among users and a negative association with the brand ensues, ultimately leading to direct losses.
To address the importance of load testing for web services like these, business owners must understand its necessity. Load testing is indispensable for several reasons:
- Ensuring continuous operation of the online product under all circumstances.
- Planning for scalability by preparing the website or app for increased demand.
- Identifying bottlenecks that hinder performance as load increases.
- Guaranteeing the ability to handle substantial volumes of traffic for online businesses.
- Proactively conducting load tests at appropriate intervals to mitigate potential issues.
By recognizing these reasons, businesses can proactively conduct load testing to safeguard against adverse scenarios and maintain optimal performance of their online products.
Now, what exactly is a load test?
Load testing entails applying pressure to software and gauging its response to that pressure. Unlike other testing methodologies, load testing requires predetermined and known stress levels. It typically simulates scenarios where numerous users access the software simultaneously. The primary objective is not only to address identified vulnerabilities within the tested software but also to develop scripts for subsequent load tests and establish control over the system. Through this process, the behavior of the system under test is comprehended, rendering it more predictable under both normal and abnormal conditions.
Identifying bottlenecks during load testing is crucial as it enables necessary measures to be taken to stabilize the overall performance of the application. By pinpointing areas of strain or limitation, developers can implement solutions to enhance the efficiency and reliability of the app.
What do the stress test results show?
The following information is displayed.
– Number of users who can access the software simultaneously.
– Software scalability, i.e.
How many more users can use it?
– Software response to peak protocol.
– Evaluate your infrastructure and determine if it is sufficient to maintain your software. In other words, load testing clearly defines the bottlenecks of your product.
What does load testing give me as the product owner?
Establish a robust and reliable infrastructure. Avoid recurrent failed login attempts. Prior to launching your product, grasp its limitations and the factors behind those limitations. Assess the components contributing to the swift and sustainable performance of your app or web service. Gain insight into product capacities, loads, volumes, and other relevant metrics. Monitor system response time and data transmission levels. Determine if your system can effectively serve users during peak traffic periods, such as advertising campaigns. Identify any non-functional requirements for the project, if not already defined. Addressing issues in your app early on is far more cost-effective than rectifying them during inactive periods, potentially at the expense of your customer base.
When should I run the load test?
One prevalent timing strategy for load testing involves conducting tests early in the pre-release phase, utilizing more manageable configurations. This method enables the discovery of crucial insights into system behavior without necessitating a specific user count. Following completion of the tests, any discrepancies between app performance and its objectives, as well as any bottlenecks, become evident. Employing a series of straightforward tests at various deployment stages proves suitable in this scenario.The benefit of this approach lies in its capacity to identify and rectify errors at an early stage, potentially resulting in cost savings.
Alternatively, another approach involves simulating real-life scenarios, necessitating meticulous configuration to mirror reality as closely as possible. However, deploying such an environment requires considerable time and effort. This approach, while offering a more accurate representation of real-world conditions, may prolong schedules due to its complexity and resource-intensive nature.
STAGES OF STRESS TEST
Let’s move on to the stages of stress testing:
- Analysis of non-functional requirements. Unlike functional testing, verifying the performance of specific functions (like “place an order” in the delivery app or “call a driver” in a taxi app), non-functional testing evaluates the performance of a system in part of its capacity, load, volume, and stress.
- Collection of the information about business processes that should be tested regarding load.
- Determine success criteria for a load test.
- Preparing the software test stand to enable the load script corresponding to the product load. The cost of this software stand is approximately equal to one copy of the app’s infrastructure, summed with the load test infrastructure depending on traffic.
- Preparing the infrastructure for the tool that launches load scripts.
- Preparing the infrastructure to assemble both hardware and system metrics.
- Preparing the load-scripts.
- Load testing itself. It may last from 1-2 hours to 6-9 hours. In critical cases, for example, at risk of release failure, it may take extended time and have the testing mode on 24/7.
- Obtaining and interpreting test results.
What other aspects of load tests should I pay attention to?
➜ Although load testing includes some parts of performance testing, it is not performance testing. The latter inspects the overall performance of the software and initiates extensive testing. As explained above, load testing is about how the software behaves when the system is subjected to user load. Other performance test types include endurance, load, scalability, volume, capacity, and peak tests.
➜ Stress testing reflects information both qualitatively and quantitatively. Once you have your productivity metrics on the table, you can align them with the non-functional requirements set out in your plan. If the team is not satisfied with the results, they can immediately fix key metrics and follow the test script to perform similar tests in the future. A series of tests is a way to dynamically control app development.
➜ Through testing, identify the limits of your app’s scaling and the components that prevent this scaling. These failures include memory leaks and a limited number of connection queues. In this way, the tested app may return technical errors such as: These include a lack of available connections, timeout errors that reflect disconnections between services, and processing errors in intermediate services that work fine under no load. In the worst case, a short-term system failure may occur.
➜ This recommendation is especially true for newly launched services that have begun commercial endeavors. If the software under test experiences peak loads (such as holidays or marketing campaigns), we recommend that you communicate with your engineers.
➜ For established products that demonstrate sustained performance under normal loads, testing at higher loads is recommended.
➜ With a load test already started and the first steps performed, you can automate the rest of the process and use it for repeated testing in the next release.
➜ If your system is under high load, we recommend paying attention to the hardware metrics and network of your test infrastructure. Connections can expire and traffic volume and user actions can impact processor/memory capacity.
➜ When writing load scripts, you need to accommodate a wide range of data selections. The script can retrieve data from known data files or generate random values within the range available to the testbench. This is important because accessing a wide range of data ensures that the database covers most requests, whereas repeated requests during a load test rely only on the cache.
➜ The amount of data in your test bench should correspond to your production environment. This point especially needs to be taken into account by engineers. When the amount of data is small, requests are more likely to complete faster than under natural conditions.
➜ Diagrams can show that the system is sustainable. However, practice does not prove this. In this case, additional software analysis is required.
How should you interpret your test results and act on them?
Interpreting test results is done in two steps:
- Analysis of the obtained metrics and comparison with the requirements of previous tests or with the corresponding metrics. This phase focuses on throughput capacity, response time, traffic volume, and error levels. If the response time is too long and an error is discovered, it does not mean the test has failed. Percentile values are applied when analyzing load metrics. Why is this measurement important in test analysis? It allows app owners to estimate how many users will enjoy a given response time and whether it will meet their goals. Percentile refers to the percentage of values that do not exceed a particular value. For example, a 50th percentile response time of 5 seconds means that half of the transactions (or 50% of transactions) take 5 seconds or more, or longer. Once these metrics are set, the engineer will set certain thresholds for the next test, and values outside this range will not be allowed to pass.
- Analyze system metrics for services and hardware integrated into your app. This phase primarily focuses on metrics that are already entered into Elasticsearch or InfluxDB for further visualization. Load testing can highlight key metrics and information when bottlenecks are approaching or indicate complete system degradation. This allows production engineers to set alerts based on relevant areas and quickly respond to issues as they arise.
How bottlenecks affect the productivity of your project
Just like the neck of a bottle slows down the water flow, a single component can be a notable hindrance in the whole workflow of your system. That’s why it’s common to refer to these elements as bottlenecks.
If you anticipate constant growth of visitors and users, the unexpected failure to handle app traffic during one or a few releases can cause financial and reputational loss. So bottlenecks restrain the scaling of an app and prevent its evolution into a more prosperous business enterprise or a popular online tool.
Which components can impede further growth?
All software systems comprise three essential levels, which may include remote hardware systems, software frameworks, and network systems. Cloud-based hardware encompasses components such as databases and web page storage. The software framework is responsible for rendering and displaying the website, while also facilitating data transmission. The seamless operation of web applications is ensured through the effective coordination of these three layers. Any component with the lowest capacity, speed, or throughput becomes a bottleneck, potentially causing delays in system performance. Common bottleneck factors include CPU usage, memory usage, disk usage, and network latency, which are identified through rigorous testing. In a recent stress testing scenario involving an educational app, a client sought to ascertain whether the service could support 80,000 concurrent users. Additionally, they wanted to determine the maximum user capacity of the system. The educational web application in question aids users in accurately identifying and pronouncing names, a vital aspect in diverse university environments. Hence, it’s crucial for the app to swiftly return search results and display names.The primary objective of the stress testing was to evaluate the authentication systems and uncover any potential limitations. Users log into the app using their credentials, triggering a signal to the API. The focus was on obtaining precise resilience information rather than providing a rough assessment of system resilience. Further testing and monitoring will be conducted based on the insights gleaned from this initial assessment.
Tools
We deployed the stack on K6 tools, which supports JavaScript (JS) as a universal framework and programming language. This means involving all team members in the evaluation and voting process. K6 puts more stress on the system, but uses fewer resources to keep it working. These benefits make K6 a cost-effective solution and are why we chose this option above all. Data was stored using the open source database Influx DB. I used Grafana for visualization. The app’s infrastructure metrics were processed using Kubernetes, New Relic, and Sumo logic (the latter used to store and collect data).
Stages of tests
Technical aspects are agreed with the customer: Use of machinery for testing. This is deployed on the customer side and represented by EC2. Test time – Night time was chosen to test the production environment.
Creating scripts and goals: Preparing load test scenarios, Preparing tools and test environments, Developing goals and milestones with customers, Identifying non-functional requirements. Comparison data with application metrics is based on visualization. In the potential diagram, you can see how well the system is handling the load. If the indicator is close to the “ceiling”, it indicates a possible bottleneck. If your hardware capacity doesn’t reach its limit as the load increases, you know it’s sustainable and can handle significant throughput. A pod is a small execution unit in Kubernetes. Finally, system parameters were received. 50 responses per second was the maximum capacity. Before testing, the understanding of how the system reacted to third-party system integration was not the same. So the test gave us new information to act on. After increasing the number of copies of the app tested for browser extensions, we were able to determine that the bottleneck was the browser extension.
The answers to the main questions were after that documented:
- This service cannot manage 80,000 users at the same time.
- The system can manage approximately 200,000 users, but it takes an hour for the system to process this.
- It is also now possible to identify at what stage and under what conditions a system begins to deteriorate.
- Making recommendations regarding the tested services. The performed tests served as the basis for further research.
The metrics we gathered enabled us to establish service level agreements (SLAs) and define benchmarks for system capacity, delineating thresholds for what constitutes satisfactory and unsatisfactory performance.
In conclusion, stress testing offers insights that cannot be confined within a narrow scope. It serves as a comprehensive analysis tool, offering deeper insights into the internal capacity of the system. This valuable data forms the foundation for ongoing monitoring of system performance and identification of potential vulnerabilities. Given the absence of a perfect, one-size-fits-all approach to load testing, engaging an experienced developer from the outset is crucial for achieving successful outcomes. Their expertise ensures that the testing process is executed effectively, yielding actionable insights and optimizing system performance.
Related Articles
This article discusses performance bottlenecks in databases and software services, explaining the issues and how to find and fix them.
When seeking a new IT service provider, it's crucial to consider evaluation criteria to find the right match.