5 ms·

Show HN: K6 – developer-centric load testing using Go and JavaScript

101 points by rlonn 9y ago

rlonn 9y ago
k6 is our attempt at creating a modern, developer/devops-centric load testing tool suitable for automation. It is released as open source on Github, scriptable in JavaScript (ES6), and built on a Golang core that provides good speed and concurrency/scalability.
goatofhehills 9y ago
Does this support websocket?
- rlonn 9y ago
  No, there is currently no support for websockets. We have been asked for this a couple of times though, and are considering bumping up the priority for a websockets implementation. As of now, k6 can do standard http, https (and it also supports HTTP/2)
  - goatofhehills 9y ago
    Thanks for the info! Would love ws suppprt!
    - rlonn 9y ago
      If you're interested, here is the Github issue for Websockets support in k6: https://github.com/loadimpact/k6/issues/178 https://github.com/loadimpact/k6/issues/178 We'd really appreciate any comments/feedback you may have on exactly how it should be implemented.
  - softwarelimits 9y ago
    Yes, websockets please! Great! Thank you!
    - rlonn 9y ago
      We just put up a $1000 bounty for Websockets support for k6: https://www.bountysource.com/issues/43973195-implement-support-for-websockets https://www.bountysource.com/issues/43973195-implement-suppo...
robbles 9y ago
This looks incredible for work I was previously using Tsung / Gatling for. One request: please allow the output of either a) a CSV log of every request and it's timing or b) a configurable histogram of request timing, not just some percentiles and averages. That info is pretty much required for doing any more advanced analysis beyond "the service is fast enough". https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta does this really well (but doesn't have the scripting abilities of this tool).
- rlonn 9y ago
  Thanks for the nice words! If you see anything missing, don't hesitate to create an issue about it on Github (https://github.com/loadimpact/k6 https://github.com/loadimpact/k6). We're trying really hard to build the things people ask for. About CSV output though, we have JSON output with single request timings, see https://docs.k6.io/docs/results-output#section-json-output https://docs.k6.io/docs/results-output#section-json-output Is JSON useful to you, or do you prefer CSV?
  - robbles 9y ago
    That looks like what I'm looking for, thanks! Is there documentation of the format of those JSON logs? CSV would probably be a useful addition just for ease of ingestion into analysis tools and databases. Once I get a chance to test this out, I'll definitely report any issues through GH.
    - rlonn 9y ago
      I'll update the docs with some info on the JSON format first thing on monday! If you should happen to try out k6 in the meantime, most of the JSON contents should be recognizable/understandable just by looking at the output, if you know what metrics k6 collects (which is documented on docs.k6.io). You can try running with e.g. 2 VUs for 2 iterations, to get a tiny amount of output data but something that looks like a real load test: k6 run --vus 2 --iterations 2 --output json=output.json script.js
brunoluiz 9y ago
At work, we had to do some load testing and we used Vegeta: https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta Had anyone used both to point out the main differences from k6?
- rlonn 9y ago
  I am of course biased, but I have used Vegeta. I did a writeup on it, and a bunch of other load testing tools in this article: http://blog.loadimpact.com/open-source-load-testing-tool-review http://blog.loadimpact.com/open-source-load-testing-tool-rev... The single biggest difference is that k6 lets you write test cases in real code (JavaScript), while Vegeta will only let you supply a static list of URLs to hit. Vegeta has some things k6 does not have, to be fair (like constant RPS-rate load), but k6 was created to address what we saw as missing in Vegeta and most of the other existing tools. We think developers want to write code to test things, so we created a tool that lets you use JavaScript + a scripting API designed for load testing and automation.
lobster_johnson 9y ago
Any plans to support distributed testing (e.g. like Tsung) or at least the ability to aggregate results once you've run it manually on multiple nodes?
- rlonn 9y ago
  Yeah, we have this issue: https://github.com/loadimpact/k6/issues/140 https://github.com/loadimpact/k6/issues/140 But you can actually aggregate results already, if you use InfluxDB as results storage: you can have multiple k6 instances push results data to the same InfluxDB instance. Results are just time series samples, so there are basically no bad effects if you do this - you get all the results in a single db, with correct time stamps. The only difficulty lies in orchestrating the multiple k6 instances so they're in sync and you get the traffic levels you want (i.e. they ramp up/down at the same time). But for a simple ramp configuration, or a config where you run at a stable traffic level, it shouldn't be a problem.
  - lobster_johnson 9y ago
    Super, thanks. Didn't realize it had InfluxDB support.
KirinDave 9y ago
Anything that uses Golang is by definition not developer centric.
- pvg 9y ago
  That's one hell of a persnickety definition.
  - KirinDave 9y ago
    I knew it would be unpopular. I get a few u popular posts a year. But I am very, very irritated by Golang while I am working with it on a daily basis. It's entire design is informed by assuming that no developer should understand the codebase, but should instead read it over and over. Symmetrically, no insight can be offered in the code, only a rote transcription of an inductive design. The net result is a language hostile to both those writing and those reading the code. No shortcuts are offered or taken. All error handling must be done manually. All data structures must be reimplemented over and over with different test suites for each one.
    - pvg 9y ago
      Heh. You can't say I was wrong calling you persnickety, at least!
      - KirinDave 9y ago
        I guess I'm gonna have to write a blog post about how hateful Go is.
- raverbashing 9y ago
  This website is written in Arc (Lisp), I guess it's not "developer centric" as well
  - KirinDave 9y ago
    What?
tpetry 9y ago
The params of http.get are not documented. Is it possible to specify a random timeout for every http request to simulate long poolling?
- rlonn 9y ago
  Thanks, I'll make sure the "params" object gets documented also. Currently, there is not so much you can set per HTTP request. You can basically add HTTP headers and you can also add tags that lets you single out specific results based on those tags (e.g. when using thresholds or when plotting graphs of results). So - HTTP request timeouts can not be changed at the moment, but this is something I think would be very simple to implement. If you open an issue about it I'll see what we can do to get it implemented quickly.

101 points by rlonn 9y ago

k6 is our attempt at creating a modern, developer/devops-centric load testing tool suitable for automation. It is released as open source on Github, scriptable in JavaScript (ES6), and built on a Golang core that provides good speed and concurrency/scalability.

goatofhehills 9y ago

Does this support websocket?

rlonn 9y ago
No, there is currently no support for websockets. We have been asked for this a couple of times though, and are considering bumping up the priority for a websockets implementation. As of now, k6 can do standard http, https (and it also supports HTTP/2)
- goatofhehills 9y ago
  Thanks for the info! Would love ws suppprt!
  - rlonn 9y ago
    If you're interested, here is the Github issue for Websockets support in k6: https://github.com/loadimpact/k6/issues/178 https://github.com/loadimpact/k6/issues/178 We'd really appreciate any comments/feedback you may have on exactly how it should be implemented.
- softwarelimits 9y ago
  Yes, websockets please! Great! Thank you!
  - rlonn 9y ago
    We just put up a $1000 bounty for Websockets support for k6: https://www.bountysource.com/issues/43973195-implement-support-for-websockets https://www.bountysource.com/issues/43973195-implement-suppo...

robbles 9y ago

This looks incredible for work I was previously using Tsung / Gatling for. One request: please allow the output of either a) a CSV log of every request and it's timing or b) a configurable histogram of request timing, not just some percentiles and averages. That info is pretty much required for doing any more advanced analysis beyond "the service is fast enough". https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta does this really well (but doesn't have the scripting abilities of this tool).

rlonn 9y ago
Thanks for the nice words! If you see anything missing, don't hesitate to create an issue about it on Github (https://github.com/loadimpact/k6 https://github.com/loadimpact/k6). We're trying really hard to build the things people ask for. About CSV output though, we have JSON output with single request timings, see https://docs.k6.io/docs/results-output#section-json-output https://docs.k6.io/docs/results-output#section-json-output Is JSON useful to you, or do you prefer CSV?
- robbles 9y ago
  That looks like what I'm looking for, thanks! Is there documentation of the format of those JSON logs? CSV would probably be a useful addition just for ease of ingestion into analysis tools and databases. Once I get a chance to test this out, I'll definitely report any issues through GH.
  - rlonn 9y ago
    I'll update the docs with some info on the JSON format first thing on monday! If you should happen to try out k6 in the meantime, most of the JSON contents should be recognizable/understandable just by looking at the output, if you know what metrics k6 collects (which is documented on docs.k6.io). You can try running with e.g. 2 VUs for 2 iterations, to get a tiny amount of output data but something that looks like a real load test: k6 run --vus 2 --iterations 2 --output json=output.json script.js

brunoluiz 9y ago

At work, we had to do some load testing and we used Vegeta: https://github.com/tsenart/vegeta https://github.com/tsenart/vegeta Had anyone used both to point out the main differences from k6?

rlonn 9y ago
I am of course biased, but I have used Vegeta. I did a writeup on it, and a bunch of other load testing tools in this article: http://blog.loadimpact.com/open-source-load-testing-tool-review http://blog.loadimpact.com/open-source-load-testing-tool-rev... The single biggest difference is that k6 lets you write test cases in real code (JavaScript), while Vegeta will only let you supply a static list of URLs to hit. Vegeta has some things k6 does not have, to be fair (like constant RPS-rate load), but k6 was created to address what we saw as missing in Vegeta and most of the other existing tools. We think developers want to write code to test things, so we created a tool that lets you use JavaScript + a scripting API designed for load testing and automation.

lobster_johnson 9y ago

Any plans to support distributed testing (e.g. like Tsung) or at least the ability to aggregate results once you've run it manually on multiple nodes?

rlonn 9y ago
Yeah, we have this issue: https://github.com/loadimpact/k6/issues/140 https://github.com/loadimpact/k6/issues/140 But you can actually aggregate results already, if you use InfluxDB as results storage: you can have multiple k6 instances push results data to the same InfluxDB instance. Results are just time series samples, so there are basically no bad effects if you do this - you get all the results in a single db, with correct time stamps. The only difficulty lies in orchestrating the multiple k6 instances so they're in sync and you get the traffic levels you want (i.e. they ramp up/down at the same time). But for a simple ramp configuration, or a config where you run at a stable traffic level, it shouldn't be a problem.
- lobster_johnson 9y ago
  Super, thanks. Didn't realize it had InfluxDB support.

KirinDave 9y ago

Anything that uses Golang is by definition not developer centric.

pvg 9y ago
That's one hell of a persnickety definition.
- KirinDave 9y ago
  I knew it would be unpopular. I get a few u popular posts a year. But I am very, very irritated by Golang while I am working with it on a daily basis. It's entire design is informed by assuming that no developer should understand the codebase, but should instead read it over and over. Symmetrically, no insight can be offered in the code, only a rote transcription of an inductive design. The net result is a language hostile to both those writing and those reading the code. No shortcuts are offered or taken. All error handling must be done manually. All data structures must be reimplemented over and over with different test suites for each one.
  - pvg 9y ago
    Heh. You can't say I was wrong calling you persnickety, at least!
    - KirinDave 9y ago
      I guess I'm gonna have to write a blog post about how hateful Go is.
raverbashing 9y ago
This website is written in Arc (Lisp), I guess it's not "developer centric" as well
- KirinDave 9y ago
  What?

tpetry 9y ago

The params of http.get are not documented. Is it possible to specify a random timeout for every http request to simulate long poolling?

rlonn 9y ago
Thanks, I'll make sure the "params" object gets documented also. Currently, there is not so much you can set per HTTP request. You can basically add HTTP headers and you can also add tags that lets you single out specific results based on those tags (e.g. when using thresholds or when plotting graphs of results). So - HTTP request timeouts can not be changed at the moment, but this is something I think would be very simple to implement. If you open an issue about it I'll see what we can do to get it implemented quickly.