8 ms·

Ask HN: Which load testing tool do you use? Any recommendations?

51 points by symbolepro 9y ago

mtmail 9y ago
For quick bursts the free plan of https://loader.io/ https://loader.io/ is pleasant to work with. For more complex setups have a look at https://locust.io/ https://locust.io/
- rlonn 9y ago
  Locust has a very nice UX for developers, but it is very slow and produces inaccurate response time measurements
maxdemarzi 9y ago
I like to use https://gatling.io/ https://gatling.io/ for my load testing. It's super easy and you get pretty charts you can hand off to management.
- willejs 9y ago
  Gatling is great, and the best thing to use if you want performance and fine grained analysis. It's easy to use, great html output with graphs. You can also output to a bunch of other formats, and easily break out into scala. Don't let the scala put you off though, its mostly a really simple and clean dsl. The docs are good too!
- hayden592 9y ago
  +1 for Gatling. It can also output junit reports if that’s something you need too
- fcarraldo 9y ago
  Another +1 for Gatling. Super easy to use, and maybe more importantly, super easy to read the code later and grok exactly what it does. Scales well, too.
roadman 9y ago
I like to use: - https://en.wikipedia.org/wiki/Tsung https://en.wikipedia.org/wiki/Tsung - https://k6.io https://k6.io.
- gregoriol 9y ago
  Tsung definitely! Harder to use at first, but amazingly powerful
  - rlonn 9y ago
    Tsung is very good - fast and w lots of features. Its main drawback, IMO, is the XML-based config/user scenarios.
- rlonn 9y ago
  Thanks for recommending k6! Disclaimer: I'm involved in the k6 project, so quite biased :) I think k6 is the best tool available if you're a developer who wants to automate load testing. But I also actually wrote a review article comparing 11 different open source load testing tools - http://blog.loadimpact.com/open-source-load-testing-tool-review http://blog.loadimpact.com/open-source-load-testing-tool-rev...
ericb 9y ago
If you are interested in reusing your functional test libraries to run load tests, drop me a line at hello@browserup.com. We have a solution and are looking to do pilots.
davewasthere 9y ago
These are the ones I've used recently. Primarily a Windows/.Net shop, so there's a lean towards tools that are scriptable, and don't require Java. ;) Bombardier - fast cross-platform, written in Go. https://github.com/codesenberg/bombardier https://github.com/codesenberg/bombardier WCAT - quite a manual process to craft requests, but great to orchestrate several clients to hammer your server if needed. https://www.iis.net/downloads/community/2007/05/wcat-63-x64 https://www.iis.net/downloads/community/2007/05/wcat-63-x64 West Wind WebSurge - Excellent tool, Very GUI focused (but has a command-line option) very easy to copy/create requests from fiddler. Approx $900USD site license. https://websurge.west-wind.com/ https://websurge.west-wind.com/ Netling - Very nice open source project with gui/console. https://github.com/hallatore/Netling https://github.com/hallatore/Netling
social_quotient 9y ago
For large scale tests we run https://goad.io https://goad.io on lambdas . We used this a lot and can confirm that most APIs can be well stressed with it. For quick tests we use loadimpact.com
joshuamcginnis 9y ago
I like the Apache Benchmarking tool: https://httpd.apache.org/docs/2.4/programs/ab.html https://httpd.apache.org/docs/2.4/programs/ab.html
- nkristoffersen 9y ago
  I like ab as well. Simple and tells me if my app can handle a load. Exactly what I want to know. I do wish it was easier to see what the failed requests looked like though.
- zer00eyz 9y ago
  I'll second this. If you need quick and dirty this is THE tool to use. I can think of 1/2 a dozen times where this has literally saved the day.
abhi_kr 9y ago
I use Vegeta https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta whenever I need to quickly test out limits of API endpoints. Besides that, artillery also looks promising https://github.com/shoreditch-ops/artillery https://github.com/shoreditch-ops/artillery
bipinvk 9y ago
For web applications try TruLoad, this is a codeless load testing tool that executes user-like workflows for automated testing https://cloudqa.io/website-load-test/ https://cloudqa.io/website-load-test/
rlonn 9y ago
Here is my take on the most common open source tools: - Gatling: https://gatling.io/ https://gatling.io/ - Great tool. Fast, full-featured, flexible, well documented. Main drawback is that you script it in Scala... UX for automation purposes could be better also. - Tsung: http://tsung.erlang-projects.org/ http://tsung.erlang-projects.org/ - Also great. Very fast, scalable, many features. Main drawback is the XML-based DSL (Domain Specific Language) that is somewhat of a pain to use. - Vegeta: https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta - Good for simple testing of API end points with a fixed RPS rate. No scripting capability. - Apachebench: https://httpd.apache.org/docs/2.4/programs/ab.html https://httpd.apache.org/docs/2.4/programs/ab.html - Single-threaded but very, very fast so will outperform many tools that can use multiple CPU cores. Best tool around if all you want to do is hit one single, static URL and get results printed on screen. Lacks scripting. - Jmeter: http://jmeter.apache.org/ http://jmeter.apache.org/ - Very fast, tons of features, but painful UX (especially for an automation workflow). Biggest community with the most plugins/extensions/whatnot. - Siege: https://www.joedog.org/siege-home/ https://www.joedog.org/siege-home/ - Moderately fast but doesn't scale. Buggy and will crash regularly. Large measurement error. Inconsistent UI. Only reason to use is that it is an alternative to Apachebench if you want to hit a list of URLs, rather than just a single URL. - Artillery: https://artillery.io/ https://artillery.io/ - Great UX for automation. However, it lacks scripting capabilities, is slow and single-threaded and introduces a large measurement error. - Locust: https://locust.io/ https://locust.io/ - Great scripting capabilities (in pure Python), with a very nice API. However, it is the slowest tool I have seen and introduces the most measurement error of any tool also. - Wrk: https://github.com/wg/wrk https://github.com/wg/wrk - Fastest tool in the universe. About 25x faster than Locust. 3x faster than Jmeter. Scriptable in Lua. Drawbacks are limited output options/reporting and a scripting API that is callback-based, so painful to use for scripting user scenario flows. - The Grinder: http://grinder.sourceforge.net/ http://grinder.sourceforge.net/ - Fast, scriptable in Jython (Python dialect) with a nice API. Main drawback is that it is an almost dead open source project - very few updates in recent years. - k6: https://k6.io/ https://k6.io/ (bias warning: I am involved in the k6 project) - The B3ST tool!! ;) Anyway... it is fast, scriptable in Javascript (ES6), has a very nice UX and scripting API and works for both functional and performance testing (and is excellent for automation). Good docs too (https://docs.k6.io https://docs.k6.io). General advice: - If you're a Java-centric shop, take a look at Jmeter or Gatling, or perhaps Siege (all are Java apps) and you'll feel at home. You should probably start with Gatling, because it is more modern than Jmeter, with better UX, and Siege is kind of on its way out. - If you want to automate your load tests, want to test an API, and prefer to write test cases in code rather than some limited DSL, then I think k6 is the tool you should look at. Only reason not to, is if you don't want to use Javascript. - If you want to run complex, manual load tests of a web site (e.g. record user interactions and then simulate the same behaviour in a load test) you should look at Gatling or Jmeter primarily. Perhaps Tsung.

51 points by symbolepro 9y ago

For quick bursts the free plan of https://loader.io/ https://loader.io/ is pleasant to work with. For more complex setups have a look at https://locust.io/ https://locust.io/

rlonn 9y ago
Locust has a very nice UX for developers, but it is very slow and produces inaccurate response time measurements

maxdemarzi 9y ago

I like to use https://gatling.io/ https://gatling.io/ for my load testing. It's super easy and you get pretty charts you can hand off to management.

willejs 9y ago
Gatling is great, and the best thing to use if you want performance and fine grained analysis. It's easy to use, great html output with graphs. You can also output to a bunch of other formats, and easily break out into scala. Don't let the scala put you off though, its mostly a really simple and clean dsl. The docs are good too!
hayden592 9y ago
+1 for Gatling. It can also output junit reports if that’s something you need too
fcarraldo 9y ago
Another +1 for Gatling. Super easy to use, and maybe more importantly, super easy to read the code later and grok exactly what it does. Scales well, too.

roadman 9y ago

I like to use: - https://en.wikipedia.org/wiki/Tsung https://en.wikipedia.org/wiki/Tsung - https://k6.io https://k6.io.

gregoriol 9y ago
Tsung definitely! Harder to use at first, but amazingly powerful
- rlonn 9y ago
  Tsung is very good - fast and w lots of features. Its main drawback, IMO, is the XML-based config/user scenarios.
rlonn 9y ago
Thanks for recommending k6! Disclaimer: I'm involved in the k6 project, so quite biased :) I think k6 is the best tool available if you're a developer who wants to automate load testing. But I also actually wrote a review article comparing 11 different open source load testing tools - http://blog.loadimpact.com/open-source-load-testing-tool-review http://blog.loadimpact.com/open-source-load-testing-tool-rev...

ericb 9y ago

If you are interested in reusing your functional test libraries to run load tests, drop me a line at hello@browserup.com. We have a solution and are looking to do pilots.

davewasthere 9y ago

These are the ones I've used recently. Primarily a Windows/.Net shop, so there's a lean towards tools that are scriptable, and don't require Java. ;) Bombardier - fast cross-platform, written in Go. https://github.com/codesenberg/bombardier https://github.com/codesenberg/bombardier WCAT - quite a manual process to craft requests, but great to orchestrate several clients to hammer your server if needed. https://www.iis.net/downloads/community/2007/05/wcat-63-x64 https://www.iis.net/downloads/community/2007/05/wcat-63-x64 West Wind WebSurge - Excellent tool, Very GUI focused (but has a command-line option) very easy to copy/create requests from fiddler. Approx $900USD site license. https://websurge.west-wind.com/ https://websurge.west-wind.com/ Netling - Very nice open source project with gui/console. https://github.com/hallatore/Netling https://github.com/hallatore/Netling

social_quotient 9y ago

For large scale tests we run https://goad.io https://goad.io on lambdas . We used this a lot and can confirm that most APIs can be well stressed with it. For quick tests we use loadimpact.com

joshuamcginnis 9y ago

I like the Apache Benchmarking tool: https://httpd.apache.org/docs/2.4/programs/ab.html https://httpd.apache.org/docs/2.4/programs/ab.html

nkristoffersen 9y ago
I like ab as well. Simple and tells me if my app can handle a load. Exactly what I want to know. I do wish it was easier to see what the failed requests looked like though.
zer00eyz 9y ago
I'll second this. If you need quick and dirty this is THE tool to use. I can think of 1/2 a dozen times where this has literally saved the day.

abhi_kr 9y ago

I use Vegeta https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta whenever I need to quickly test out limits of API endpoints. Besides that, artillery also looks promising https://github.com/shoreditch-ops/artillery https://github.com/shoreditch-ops/artillery

bipinvk 9y ago

For web applications try TruLoad, this is a codeless load testing tool that executes user-like workflows for automated testing https://cloudqa.io/website-load-test/ https://cloudqa.io/website-load-test/

rlonn 9y ago

Here is my take on the most common open source tools: - Gatling: https://gatling.io/ https://gatling.io/ - Great tool. Fast, full-featured, flexible, well documented. Main drawback is that you script it in Scala... UX for automation purposes could be better also. - Tsung: http://tsung.erlang-projects.org/ http://tsung.erlang-projects.org/ - Also great. Very fast, scalable, many features. Main drawback is the XML-based DSL (Domain Specific Language) that is somewhat of a pain to use. - Vegeta: https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta - Good for simple testing of API end points with a fixed RPS rate. No scripting capability. - Apachebench: https://httpd.apache.org/docs/2.4/programs/ab.html https://httpd.apache.org/docs/2.4/programs/ab.html - Single-threaded but very, very fast so will outperform many tools that can use multiple CPU cores. Best tool around if all you want to do is hit one single, static URL and get results printed on screen. Lacks scripting. - Jmeter: http://jmeter.apache.org/ http://jmeter.apache.org/ - Very fast, tons of features, but painful UX (especially for an automation workflow). Biggest community with the most plugins/extensions/whatnot. - Siege: https://www.joedog.org/siege-home/ https://www.joedog.org/siege-home/ - Moderately fast but doesn't scale. Buggy and will crash regularly. Large measurement error. Inconsistent UI. Only reason to use is that it is an alternative to Apachebench if you want to hit a list of URLs, rather than just a single URL. - Artillery: https://artillery.io/ https://artillery.io/ - Great UX for automation. However, it lacks scripting capabilities, is slow and single-threaded and introduces a large measurement error. - Locust: https://locust.io/ https://locust.io/ - Great scripting capabilities (in pure Python), with a very nice API. However, it is the slowest tool I have seen and introduces the most measurement error of any tool also. - Wrk: https://github.com/wg/wrk https://github.com/wg/wrk - Fastest tool in the universe. About 25x faster than Locust. 3x faster than Jmeter. Scriptable in Lua. Drawbacks are limited output options/reporting and a scripting API that is callback-based, so painful to use for scripting user scenario flows. - The Grinder: http://grinder.sourceforge.net/ http://grinder.sourceforge.net/ - Fast, scriptable in Jython (Python dialect) with a nice API. Main drawback is that it is an almost dead open source project - very few updates in recent years. - k6: https://k6.io/ https://k6.io/ (bias warning: I am involved in the k6 project) - The B3ST tool!! ;) Anyway... it is fast, scriptable in Javascript (ES6), has a very nice UX and scripting API and works for both functional and performance testing (and is excellent for automation). Good docs too (https://docs.k6.io https://docs.k6.io). General advice: - If you're a Java-centric shop, take a look at Jmeter or Gatling, or perhaps Siege (all are Java apps) and you'll feel at home. You should probably start with Gatling, because it is more modern than Jmeter, with better UX, and Siege is kind of on its way out. - If you want to automate your load tests, want to test an API, and prefer to write test cases in code rather than some limited DSL, then I think k6 is the tool you should look at. Only reason not to, is if you don't want to use Javascript. - If you want to run complex, manual load tests of a web site (e.g. record user interactions and then simulate the same behaviour in a load test) you should look at Gatling or Jmeter primarily. Perhaps Tsung.