4 ms·

The JMeter load tests were conducted to test the throughput of the CDN and backend setup (browser caching was disabled). The load generating servers where hoste

The JMeter load tests were conducted to test the throughput of the CDN and backend setup (browser caching was disabled). The load generating servers where hosted in Frankfurt (Germany) where the CDN (Fastly) also has an edge location. This lead to average response time of 5ms. We experienced some hickups within JMeter causing single requests to stall up to 5 seconds. For the 10 million requests we got an 99.9th percentile of below 10ms. We couldn't quite figure out why JMeter had these hiccups, maybe those were GC pauses, maybe something else, but it was consistent across different setups and servers.

greglindahl 10y ago
One wonders about the irony of carefully avoiding bad stuff (like GC) when serving pages, and then using a test rig that has potential GC delays to test performance.
- erikwitt 10y ago
  You have a point there. I found jMeter, however, really easy to use. I could simply let it monitor my browser (via a proxy) while i clicked through the website and the checkout process to record the requests of an average user. Then I configured the checkout process to only be executed in 20% of the cases to simulate the conversion rate. Even executing the test distributed over 20 servers wasn't that hard. Which tools would you use to generate this amount of traffic?
  - greglindahl 10y ago
    In a previous life we started with ApacheBench, and then wrote our own async perl benchmarker because we wanted to generate steady hits-per-second instead of ApacheBench's N concurrent requests at a time. By now there's probably a tool out there that does what our custom tool did.
  - iampims 10y ago
    wrk2: https://github.com/giltene/wrk2 https://github.com/giltene/wrk2
    - greglindahl 10y ago
      That looks pretty good -- not only does it have a "requests per second" generator, which I wanted, but it also presents 50/90/99/99.9 percentile results, which are a must. (Website latencies are not a normal distribution so it's inappropriate to compute standard deviation.)
  - jrudolph 10y ago
    locust.io is pretty sweet, performs well and allows you to do custom scripting in python

greglindahl 10y ago

One wonders about the irony of carefully avoiding bad stuff (like GC) when serving pages, and then using a test rig that has potential GC delays to test performance.

erikwitt 10y ago
You have a point there. I found jMeter, however, really easy to use. I could simply let it monitor my browser (via a proxy) while i clicked through the website and the checkout process to record the requests of an average user. Then I configured the checkout process to only be executed in 20% of the cases to simulate the conversion rate. Even executing the test distributed over 20 servers wasn't that hard. Which tools would you use to generate this amount of traffic?
- greglindahl 10y ago
  In a previous life we started with ApacheBench, and then wrote our own async perl benchmarker because we wanted to generate steady hits-per-second instead of ApacheBench's N concurrent requests at a time. By now there's probably a tool out there that does what our custom tool did.
- iampims 10y ago
  wrk2: https://github.com/giltene/wrk2 https://github.com/giltene/wrk2
  - greglindahl 10y ago
    That looks pretty good -- not only does it have a "requests per second" generator, which I wanted, but it also presents 50/90/99/99.9 percentile results, which are a must. (Website latencies are not a normal distribution so it's inappropriate to compute standard deviation.)
- jrudolph 10y ago
  locust.io is pretty sweet, performs well and allows you to do custom scripting in python