3 ms·

Have you tried https://k6.io/ https://k6.io/ ? (Full disclosure: I'm one of the maintainers.) It allows you to write load/performance tests in JS, commit them

by imiric 3y ago

Have you tried https://k6.io/ https://k6.io/ ? (Full disclosure: I'm one of the maintainers.) It allows you to write load/performance tests in JS, commit them to your repo, easily automate them in CI, send metrics to several backends, use protocols besides HTTP, with a modern CLI, and many more features. There's also a Postman-to-k6 converter[1]. The conversion might not be perfect, but it will give you a head start. Note that the k6 philosophy is for developers to write these tests, similarly to how you write unit/integration tests, and to break the classic QA-dev cycle. I don't want to steal Hurl's thunder, it does look great, but it's limited in features compared to existing peformance testing tools, and I'd personally rather write tests in a programming language, than in a bespoke text format. [1]: https://github.com/apideck-libraries/postman-to-k6 https://github.com/apideck-libraries/postman-to-k6

gavinray 3y ago
If you maintain k6, can I log a complaint? I feel like I used to be able to run the CLI with just an URL and it would issue GET requests to it -- without needing a .js script to run Am I crazy, did this never exist? Please bring back/add this feature
- imiric 3y ago
  I think you're mistaking k6 for another tool. k6 has only supported writing tests in JS. There are several other tools that are better suited for this single URL use case: ab, wrk, even Hurl. I doubt this will ever be supported in k6, since the focus is on scripting test logic, not firing off requests to a single URL (though this is certainly possible with a simple script).
FridgeSeal 3y ago
> but it's limited in features compared to existing peformance testing tools, and I'd personally rather write tests in a programming language, than in a bespoke text format. I’d hardly call hurl’s config bespoke, it basically amounts to HTTP codes and some header stuff, which basically everyone is familiar with. From personal experience a couple of weeks ago I needed some http tests. I got a CI-ready Hurl config working on some endpoints for work sorted within like, 5 minutes of downloading it. In comparison I opened the page for K6, saw the “write tests using JS” and basically noped out immediately,
- imiric 3y ago
  That's fair, and k6 is not for everyone. JS was chosen as it's a popular language with a large web ecosystem, but the tool itself is written in Go. That said... > I got a CI-ready Hurl config working on some endpoints for work sorted within like, 5 minutes of downloading it. I'm obviously biased, but I'd argue that k6 is equally easy to get started with. Take a look at the docs[1]. > I’d hardly call hurl’s config bespoke But it is. It has a unique syntax, and no other tool uses it, AFAIK. I'm not saying it's not simple or easy to understand, but it's not as expressive as a programming language. Which is fine if your tests don't require any logic, but from experience, in order to simulate accurate real-world traffic, which you probably want when writing a load/performance test, you do need some logic, different scheduling options, a way to process and analyze the metrics, etc. I don't want to turn this into a versus battle, as there is a place for tools such as Hurl, ab, and wrk to coexist with tools like k6 and Locust. I just wanted to mention it as an option for anyone who hasn't heard about it, as any personal biases aside, I do think it's a great tool. [1]: https://k6.io/docs/ https://k6.io/docs/
  - FridgeSeal 3y ago
    > Take a look at the docs Yeah, I did they start off with JS shenanigans, including importing packages, which immediately makes me concerned I’m going to have to deal with JS packaging dumpster-fire. There’s then one line of CLI args, followed by even more JS boilerplate. > in order to simulate accurate real-world traffic, which you probably want when writing a load/performance test I think the mismatch here is that Hurl is closer to being a http testing tool, not a load-testing tool, so most of these features are unnecessary. > It has a unique syntax, and no other tool uses it, AFAIK. Taking the example from the front page: GET https://example.org https://example.org HTTP 200 [Captures] csrf_token: xpath "string(//meta[@name='_csrf_token']/@content)" Character for character, most of this is bog-standard http verbs, a url, and an html path string. The Jetbrain HTTP tool also sports very similar interface, and I believe there’s a VSCode plugin somewhere that does a similar thing.

by imiric 3y ago

gavinray 3y ago

If you maintain k6, can I log a complaint? I feel like I used to be able to run the CLI with just an URL and it would issue GET requests to it -- without needing a .js script to run Am I crazy, did this never exist? Please bring back/add this feature

imiric 3y ago
I think you're mistaking k6 for another tool. k6 has only supported writing tests in JS. There are several other tools that are better suited for this single URL use case: ab, wrk, even Hurl. I doubt this will ever be supported in k6, since the focus is on scripting test logic, not firing off requests to a single URL (though this is certainly possible with a simple script).

FridgeSeal 3y ago

> but it's limited in features compared to existing peformance testing tools, and I'd personally rather write tests in a programming language, than in a bespoke text format. I’d hardly call hurl’s config bespoke, it basically amounts to HTTP codes and some header stuff, which basically everyone is familiar with. From personal experience a couple of weeks ago I needed some http tests. I got a CI-ready Hurl config working on some endpoints for work sorted within like, 5 minutes of downloading it. In comparison I opened the page for K6, saw the “write tests using JS” and basically noped out immediately,

imiric 3y ago
That's fair, and k6 is not for everyone. JS was chosen as it's a popular language with a large web ecosystem, but the tool itself is written in Go. That said... > I got a CI-ready Hurl config working on some endpoints for work sorted within like, 5 minutes of downloading it. I'm obviously biased, but I'd argue that k6 is equally easy to get started with. Take a look at the docs[1]. > I’d hardly call hurl’s config bespoke But it is. It has a unique syntax, and no other tool uses it, AFAIK. I'm not saying it's not simple or easy to understand, but it's not as expressive as a programming language. Which is fine if your tests don't require any logic, but from experience, in order to simulate accurate real-world traffic, which you probably want when writing a load/performance test, you do need some logic, different scheduling options, a way to process and analyze the metrics, etc. I don't want to turn this into a versus battle, as there is a place for tools such as Hurl, ab, and wrk to coexist with tools like k6 and Locust. I just wanted to mention it as an option for anyone who hasn't heard about it, as any personal biases aside, I do think it's a great tool. [1]: https://k6.io/docs/ https://k6.io/docs/
- FridgeSeal 3y ago
  > Take a look at the docs Yeah, I did they start off with JS shenanigans, including importing packages, which immediately makes me concerned I’m going to have to deal with JS packaging dumpster-fire. There’s then one line of CLI args, followed by even more JS boilerplate. > in order to simulate accurate real-world traffic, which you probably want when writing a load/performance test I think the mismatch here is that Hurl is closer to being a http testing tool, not a load-testing tool, so most of these features are unnecessary. > It has a unique syntax, and no other tool uses it, AFAIK. Taking the example from the front page: GET https://example.org https://example.org HTTP 200 [Captures] csrf_token: xpath "string(//meta[@name='_csrf_token']/@content)" Character for character, most of this is bog-standard http verbs, a url, and an html path string. The Jetbrain HTTP tool also sports very similar interface, and I believe there’s a VSCode plugin somewhere that does a similar thing.