Date Published: October 27, 2020
Performance and scalability are two important attributes to examine whenever you're building a web application.
Performance consists of responsiveness (POV of a client), throughput (POV of system), and resource utilization (POV of system). This will vary by load, so when testing performance, load should be constant.
Scalability is related to an application's ability to increase load while holding performance constant by adding resources. This will vary when performance changes, so performance should be fixed when testing scalability.
In an ideal world, applications would have linear scale. That is, each additional resource should allow for the same increase in load as did the previous resource added. Essentially, to use a term from economics, your goal is to have constant returns to scale:
X in -> Y out, and nX in -> nY out, no matter how big n gets.
Testing can help you move towards linear scale by helping you identify bottlenecks in your application.
Measuring performance and scalability is important because it allows developers to know whether they are improving these metrics (or making them worse) and whether the system is production-ready, based on thresholds/goals that have been set for the application.
Whatever type of testing you're doing, it's important to keep track of how your application performs in the tests over time. Is it improving? Did the changes you made make a difference? Keep track of the information you'll need to answer these questions.
Don't measure irrelevant things - in an application without a database, for example, don't measure database communication. Pretty self explanatory.
On the flipside, better to measure too much than too little. But do keep in mind that more counters = more data that needs to be stored.
Performance Counter Categories:
These should be specific.
Applications like Visual Studio, along with some third-party plugins, provide the tools necessary for some very specific performance tests.
Performance tests... test an application's performance.
Load tests generally test performance based on user load, requests/sec, and errors/sec. These tests should also consider resource utilization - how many servers, how these servers are being used, etc. Load tests are a lot like performance tests, but load tests involve a number of simulated users all testing performance at the same time. Additionally, load tests generally involve a collection of different individual performance tests, rather than just running the same test over and over.
Think time is an estimation of the time a user spends on each page. In performance tests (especially load tests), think time is implemented as a delay between web requests. This is a useful tool, and the most realistic implementation of it is also the default, at least in Visual Studio: normal distribution of think time centered on think time recorded when test was recorded.
Two options for load pattern:
Constant Load - same number of users for the whole test
Step Load - increase number of users over the course of the test
You can also configure load tests to run from multiple machines at once. This is something you really should do, especially for larger tests.
Load testing, but you're trying to determine how the application responds at very high user loads. Essentially, you know the system will fail at that level of load, and you want to see how it fails.
Smoke Test - does the application work under normal circumstances?
Capacity Test - what, exactly, is the application's maximum load?
Endurance Test - how does the system function over an extended period of time (as it would have to in production)?
When examining the results of your tests, one of the most useful tools at your disposal is a graph. Look at the graphs for your counters in order to identify bottlenecks. Constant values are not interesting; horizontal lines probably won't tell you much. However, values that increase (or decrease) as users are added and values that spike at different times are counters you should pay attention to, because they will help you optimize the system.
One key counter that can be really useful in bottleneck identification is the requests per second counter. When using a step load pattern in a load test, the requests per second should increase as the number of users increases. At some point, though, requests per second may plateau, and this is an indication of the application's highest possible requests per second, given its current configuration.
This post has notes from the following sources:
Thanks for reading! I hope you find this and other articles here at ilyanaDev helpful! Be sure to follow me on Twitter @ilyanaDev.