We’re going to use a test app. Supposedly, this app has passed a first development iteration, and has passed unit and functional testing. With JMeter we measure its performance.
Performance testing should start early and continue through the whole dev process.
JMeter is not a browser. It doesn’t execute JavaScript!
Performance Testing
Performance is related to how fast an app executes an operation. Performance testing is about how an app or resource performs under a given load, to see its impact. It should be done after the app has passed all functional tests and we’re sure it works correctly.
Performance testing is measured in terms of:
- response time: request time + processing time + network latency.
- throughput: number of transactions (request/response) / unit of time (ms / seconds).
- reliability: how well the app detects and handles errors. number of errors / number of requests.
-
scalability: tells how well the system expands its capacity in terms of response time, throughput and reliability when additional resources are added.
- vertical scalability: adding more memory or additional CPUs.
- horizontal scalability: adding servers to a cluster.
Performance requirements
They’re are usually set up in contracts, such as:
- average / maximum response time.
- pages per second the system should be able to support.
- users per hour the system should be able to support.
Process to follow
- Design and build tests: We have to know the app and its usage patterns.
- Prepare test environment: configure a test env similar to prod.
- Run the test: validate the script and test data. Monitor server logs!
- Analyze the results
- Optimize
- Retest
Types of Performance Tests
- Increase the number of users.
- Increase the number of requests.
Almost always you should execute a smoke test first. This is a test with light load to verify the test works correctly.
A load test is a test that’s performed at a specific load level. Usually you’ll perform them at many load levels to monitor the app’s behavior.
A stress test tests the app with loads past its normal working range, to see up to which point it stays stable and responsive.
At a spike test the app is subjected to brief periods of sudden increments in the load beyond its maximum capacity, to see if the app is robust enough to work during and after the spike.
Endurance tests, where the app is subjected to load within its limits for a long duration (hours or even days) to search for memory leaks.