10 ms·

Ask HN: What are you using for http load testing?

I'm looking for something more flexible than apache ab, but not as heavy weight as JMeter. Ideally I'd like to be able to specify some random values in urls or post requests. My environment is linux/ubuntu. What are you using these days?

slipwalker 9y ago
a mix of ab and vegeta ( https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta ). More and more vegeta lately, thou.
technion 9y ago
I really like using goad. https://github.com/goadapp/goad https://github.com/goadapp/goad
- bkdonline 9y ago
  httperf.
jdelsman 9y ago
k6 by Load Impact: https://k6.io https://k6.io
hamilyon2 9y ago
One ot the best tools for load testing on ubuntu is yandex tank.
YuriNiyazov 9y ago
Siege
scrollaway 9y ago
Locust. https://locust.io https://locust.io
- saopayne 9y ago
  Plus 1 for this!
- lalwanivikas 9y ago
  +1. Our team also uses Locust.
guessmyname 9y ago
Here is a good list — https://github.com/denji/awesome-http-benchmark https://github.com/denji/awesome-http-benchmark It includes the good ones like Vegeta, Siege, Goad, Hey, Httpref, Gatling, among others.
tofflos 9y ago
Vegeta. https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta. The following post covers an approach for generating some randomness. https://thisdata.com/blog/load-testing-api-interfaces-with-go-and-vegeta https://thisdata.com/blog/load-testing-api-interfaces-with-g....
gh0zt 9y ago
Gatling (https://gatling.io https://gatling.io)
quickthrower2 9y ago
Customers
- samkone 9y ago
  +1
deleted 9y ago
[deleted]
davewasthere 9y ago
From the Microsoft world: WCAT: https://www.iis.net/downloads/community/2007/05/wcat-63-x86 https://www.iis.net/downloads/community/2007/05/wcat-63-x86 (pretty old and quite manual, but found it really effective, can control many clients distributing the load) West Wind's WebSurge: https://websurge.west-wind.com/ https://websurge.west-wind.com/ Bloody good bit of kit and very easy to user Bombadier: https://github.com/codesenberg/bombardier https://github.com/codesenberg/bombardier Very fast Go HTTP(S) benchmarking. And Netling: https://github.com/hallatore/Netling https://github.com/hallatore/Netling I just really like it.
tpaschalis 9y ago
> not as heavy weight as JMeter JMeter is indeed a tad heavier, but I find it's much more user friendly as in, you spend a couple of hours to build basic tests, or afternoon and have learned most of its functionality. It also has a variety of ways implementing this randomized user behavior [1]. https://tpaschalis.github.io/jmeter-random-user-actions/ https://tpaschalis.github.io/jmeter-random-user-actions/
geocar 9y ago
I like to use wrk[1] for quick/spot testing: * It allows me to focus on latency versus throughput independently * It has tools for testing performance on pipelined requests (very important in my space) * It supports scripting complex authentication processes (like magic headers and oauth2) * It supports scripting fuzzing (random values) for URLs and POST requests [1]: https://github.com/wg/wrk https://github.com/wg/wrk For wide-area load testing, I simply buy some advertising since that gives me cheap access to millions of simultaneous users.
- anonacct37 9y ago
  We use wrk since we are often load testing fairly simple services, occasionally with some Lua for cache busting. It's a great tool. Would love if it supported http2 since I work on CDN type stuff where that matters. I love your 2nd approach (advertising). Any pointers to how to get started?
  - geocar 9y ago
    go-wrk[1] promises HTTP/2 support and is similar to wrk. I don't have a lot of experience with it though. [1]: https://github.com/tsliwowicz/go-wrk https://github.com/tsliwowicz/go-wrk If you want to purchase advertising for this purpose, I'm happy to help with this. My contact details are linked through my profile. - Your HTTP endpoints will need to be whitelisted with traffic quality groups and creative scanners (This can take a week or so). I recommend using a separate dns domain name for this since adblockers tend to subscribe to these lists. - Your JavaScript should not be abusive. There are subjective tests for this. Generally doing an image request or some POST data of a few kb isn't going to be an issue. Some ad servers (like sizmek) download 2mb "test images" to exercise their bandwidth from some locations (like the US). - Costs vary based on device, region, the web page the ads are on, and silly things like demographic. Some of these factors are interesting to benchmarkers but others are not. Thinking about what you actually need to test can help keep your costs efficient.
- togusa2017 9y ago
  I agree. I also love wrk and it is pretty useful and allows scripting
anonacct37 9y ago
Load testing can be hard. Wrk works for me for the most part but it's not ideal for complex flows (say e-commerce trying to simulate a real browse and checkout complete with typeahead search). However the smarter apps can have problems too. I've pinned all available cores on fairly beefy machines because they are gunzipping bodies (behaving more like a real browser). If you are at a scale where you need really good testing I recommend that you think of wrk results as comparable to a unit test and something like jmeter as comparable to an integration test. It's not worth the effort for most companies but where I work we serve a decent amount of traffic and downtime costs 6 figures per minute so good performance engineering and good business overlap.
ing33k 9y ago
I use hey[1]. Shameless plug: I have developed a small tool called vex [2] ( using PHP ) for this purpose but it's not feature complete. [1] https://github.com/rakyll/hey https://github.com/rakyll/hey [2] https://github.com/vamsiikrishna/vex https://github.com/vamsiikrishna/vex
rectalogic 9y ago
Artillery via AWS lambda https://github.com/Nordstrom/serverless-artillery https://github.com/Nordstrom/serverless-artillery
scrpn 9y ago
apache benchmark (ab) is a good alternative for simple test scenarios
shifte 9y ago
You can definitely do all those things with JMeter, although the user experience is pretty naf, Gatling[3] is definitely much better at that front if you're comfortable writing Scala. When it comes to Protocol Level Testing, those two tools are still pretty much the best bang for buck, and by far the most popular open source load generators. If the target you're hitting is a web app and you want to create a more realistic load scenario without crafting individual requests, check out a tool called Flood Chrome, which uses thousands of instances of Google Chrome[1] to generate load. It uses the same scripting language as SeleniumJS, so it's quite easy to learn. Also on the topic of Flood [2], they're a very good platform for abstracting away the server orchestration and results collection you'll face when you try to scale up from 1 to N servers running either Gatling or JMeter. [1]: https://chrome.flood.io/ https://chrome.flood.io/ [2]: https://flood.io/ https://flood.io/ [3]: https://gatling.io/ https://gatling.io/
wheresvic1 9y ago
I use tsung - https://smalldata.tech/blog/2015/08/26/load-testing-using-tsung https://smalldata.tech/blog/2015/08/26/load-testing-using-ts...
umen 9y ago
Jmeter is de facto in telecom , it's far from perfect but it let you define complex flows quick , the pain is that you need to ménage the test plans on the server to run all the jmx files I wish there was full web interface on the server already
hkchad 9y ago
Bees with machineguns https://github.com/newsapps/beeswithmachineguns https://github.com/newsapps/beeswithmachineguns
lyonlim 9y ago
We use JMeter and run them on AWS load generators using redline13.com
theden 9y ago
https://gettaurus.org/ https://gettaurus.org/ + BlazeMeter (for reports)
artpar 9y ago
https://hackernoon.com/to-start-thinking-about-load-testing-e68384b24a12 https://hackernoon.com/to-start-thinking-about-load-testing-...
unrequited 9y ago
https://github.com/giltene/wrk2 https://github.com/giltene/wrk2
jxub 9y ago
HN users.

slipwalker 9y ago

a mix of ab and vegeta ( https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta ). More and more vegeta lately, thou.

technion 9y ago

I really like using goad. https://github.com/goadapp/goad https://github.com/goadapp/goad

bkdonline 9y ago
httperf.

jdelsman 9y ago

k6 by Load Impact: https://k6.io https://k6.io

hamilyon2 9y ago

One ot the best tools for load testing on ubuntu is yandex tank.

YuriNiyazov 9y ago

Siege

scrollaway 9y ago

Locust. https://locust.io https://locust.io

saopayne 9y ago
Plus 1 for this!
lalwanivikas 9y ago
+1. Our team also uses Locust.

guessmyname 9y ago

Here is a good list — https://github.com/denji/awesome-http-benchmark https://github.com/denji/awesome-http-benchmark It includes the good ones like Vegeta, Siege, Goad, Hey, Httpref, Gatling, among others.

tofflos 9y ago

Vegeta. https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta. The following post covers an approach for generating some randomness. https://thisdata.com/blog/load-testing-api-interfaces-with-go-and-vegeta https://thisdata.com/blog/load-testing-api-interfaces-with-g....

gh0zt 9y ago

Gatling (https://gatling.io https://gatling.io)

quickthrower2 9y ago

Customers

samkone 9y ago
+1

deleted 9y ago

[deleted]

davewasthere 9y ago

From the Microsoft world: WCAT: https://www.iis.net/downloads/community/2007/05/wcat-63-x86 https://www.iis.net/downloads/community/2007/05/wcat-63-x86 (pretty old and quite manual, but found it really effective, can control many clients distributing the load) West Wind's WebSurge: https://websurge.west-wind.com/ https://websurge.west-wind.com/ Bloody good bit of kit and very easy to user Bombadier: https://github.com/codesenberg/bombardier https://github.com/codesenberg/bombardier Very fast Go HTTP(S) benchmarking. And Netling: https://github.com/hallatore/Netling https://github.com/hallatore/Netling I just really like it.

tpaschalis 9y ago

> not as heavy weight as JMeter JMeter is indeed a tad heavier, but I find it's much more user friendly as in, you spend a couple of hours to build basic tests, or afternoon and have learned most of its functionality. It also has a variety of ways implementing this randomized user behavior [1]. https://tpaschalis.github.io/jmeter-random-user-actions/ https://tpaschalis.github.io/jmeter-random-user-actions/

geocar 9y ago

I like to use wrk[1] for quick/spot testing: * It allows me to focus on latency versus throughput independently * It has tools for testing performance on pipelined requests (very important in my space) * It supports scripting complex authentication processes (like magic headers and oauth2) * It supports scripting fuzzing (random values) for URLs and POST requests [1]: https://github.com/wg/wrk https://github.com/wg/wrk For wide-area load testing, I simply buy some advertising since that gives me cheap access to millions of simultaneous users.

anonacct37 9y ago
We use wrk since we are often load testing fairly simple services, occasionally with some Lua for cache busting. It's a great tool. Would love if it supported http2 since I work on CDN type stuff where that matters. I love your 2nd approach (advertising). Any pointers to how to get started?
- geocar 9y ago
  go-wrk[1] promises HTTP/2 support and is similar to wrk. I don't have a lot of experience with it though. [1]: https://github.com/tsliwowicz/go-wrk https://github.com/tsliwowicz/go-wrk If you want to purchase advertising for this purpose, I'm happy to help with this. My contact details are linked through my profile. - Your HTTP endpoints will need to be whitelisted with traffic quality groups and creative scanners (This can take a week or so). I recommend using a separate dns domain name for this since adblockers tend to subscribe to these lists. - Your JavaScript should not be abusive. There are subjective tests for this. Generally doing an image request or some POST data of a few kb isn't going to be an issue. Some ad servers (like sizmek) download 2mb "test images" to exercise their bandwidth from some locations (like the US). - Costs vary based on device, region, the web page the ads are on, and silly things like demographic. Some of these factors are interesting to benchmarkers but others are not. Thinking about what you actually need to test can help keep your costs efficient.
togusa2017 9y ago
I agree. I also love wrk and it is pretty useful and allows scripting

anonacct37 9y ago

Load testing can be hard. Wrk works for me for the most part but it's not ideal for complex flows (say e-commerce trying to simulate a real browse and checkout complete with typeahead search). However the smarter apps can have problems too. I've pinned all available cores on fairly beefy machines because they are gunzipping bodies (behaving more like a real browser). If you are at a scale where you need really good testing I recommend that you think of wrk results as comparable to a unit test and something like jmeter as comparable to an integration test. It's not worth the effort for most companies but where I work we serve a decent amount of traffic and downtime costs 6 figures per minute so good performance engineering and good business overlap.

ing33k 9y ago

I use hey[1]. Shameless plug: I have developed a small tool called vex [2] ( using PHP ) for this purpose but it's not feature complete. [1] https://github.com/rakyll/hey https://github.com/rakyll/hey [2] https://github.com/vamsiikrishna/vex https://github.com/vamsiikrishna/vex

rectalogic 9y ago

Artillery via AWS lambda https://github.com/Nordstrom/serverless-artillery https://github.com/Nordstrom/serverless-artillery

scrpn 9y ago

apache benchmark (ab) is a good alternative for simple test scenarios

shifte 9y ago

You can definitely do all those things with JMeter, although the user experience is pretty naf, Gatling[3] is definitely much better at that front if you're comfortable writing Scala. When it comes to Protocol Level Testing, those two tools are still pretty much the best bang for buck, and by far the most popular open source load generators. If the target you're hitting is a web app and you want to create a more realistic load scenario without crafting individual requests, check out a tool called Flood Chrome, which uses thousands of instances of Google Chrome[1] to generate load. It uses the same scripting language as SeleniumJS, so it's quite easy to learn. Also on the topic of Flood [2], they're a very good platform for abstracting away the server orchestration and results collection you'll face when you try to scale up from 1 to N servers running either Gatling or JMeter. [1]: https://chrome.flood.io/ https://chrome.flood.io/ [2]: https://flood.io/ https://flood.io/ [3]: https://gatling.io/ https://gatling.io/

wheresvic1 9y ago

I use tsung - https://smalldata.tech/blog/2015/08/26/load-testing-using-tsung https://smalldata.tech/blog/2015/08/26/load-testing-using-ts...

umen 9y ago

Jmeter is de facto in telecom , it's far from perfect but it let you define complex flows quick , the pain is that you need to ménage the test plans on the server to run all the jmx files I wish there was full web interface on the server already

hkchad 9y ago

Bees with machineguns https://github.com/newsapps/beeswithmachineguns https://github.com/newsapps/beeswithmachineguns

lyonlim 9y ago

We use JMeter and run them on AWS load generators using redline13.com

theden 9y ago

https://gettaurus.org/ https://gettaurus.org/ + BlazeMeter (for reports)

artpar 9y ago

https://hackernoon.com/to-start-thinking-about-load-testing-e68384b24a12 https://hackernoon.com/to-start-thinking-about-load-testing-...

unrequited 9y ago

https://github.com/giltene/wrk2 https://github.com/giltene/wrk2

jxub 9y ago

HN users.