7 ms·

Show HN: A Swiss army knife for testing HTTP from the terminal

110 points by hassy 4y ago

Hiya HN! Just released Artillery Probe - a Swiss army knife for testing HTTP from the CLI. Think mini-curl with better UX for common use-cases, plus a couple of extra features. Would love for you to try it and give some feedback! https://www.artillery.io/blog/swiss-army-knife-for-http-testing and: npm install artillery@latest What does it do? - First of all, it's a HTTP client! It does all the usual stuff you'd expect from a HTTP client... HTTP methods, request bodies, custom headers, forms, Basic Auth etc. - Got some JSON or XML back? It'll pretty-print it, and syntax highlight it for you. - It'll show you request waterfalls like this one: (inspired by httpstat) DNS Lookup | TCP Connection | SSL Handshake | Time to First Byte | Content Transfer 56ms | 14ms | 19ms | 181ms | 88ms | | | | | | 56ms | | | | 70ms | | | 89ms | | 270ms | total:358ms - JSON responses can be queried and sliced and diced with JMESPath (same syntax as AWS CLI) - no need to reach for jq. XML & HTML may be queried with a jQuery-like syntax too. - You can set expectations on the response, e.g. have the CLI check that the response is a 200, or that a certain header is set, and exit with non-zero code if not. Super handy for quick acceptance testing. We've got lots of ideas for improvements, but would love to hear what you think!

hassy 4y ago
clickable url: https://www.artillery.io/blog/swiss-army-knife-for-http-testing https://www.artillery.io/blog/swiss-army-knife-for-http-test...
jsd1982 4y ago
> npm install Sorry, lost me at this part. Have a better installation story like a single executable. Beyond that, what does this do that curl, wget, et al do not? How is your interface better than those? Is it robust in the face of errors? What http protocol versions does it support?
- hassy 4y ago
  if you’re on a Mac, Artillery has a Homebrew formula too. We have an official Docker image too. Other ways of installing are on the roadmap too (self contained tarballs and binaries). As to what it does better than curl/wget. Probe is geared towards interactive use with HTTP APIs. So you get syntax highlighting and pretty printing for JSON, built-in querying, request waterfall visualizations, and ability to set expectations on every response. HTTP/2 is supported.
  - jsd1982 4y ago
    https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... Does artillery properly escape the environment variables into json strings? I don't see how it could if you're using bash expansion in double quotes. This feature would only work with string values without spaces or special characters. Don't really see any improvement here. https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... So by embedding a json query evaluator in your http client, this means that every time I iterate on my JMESPath query I'm making a new request? Is there an offline mode? What if my Json file is 200MiB? Or if the request was not idempotent? Just saying there's good reasons why json query features are not typically built into one-shot http request tools. However, if your tool was not one-shotand you had an interactive session and cached/stored the entire result and then allow the user to interactively query it, then I could see some better value here.
    - hassy 4y ago
      Yeah so the JSON quoting part is something I’m pretty pleased with. We use a YAML parser (JSON is a subset of YAML) to parse those values, which is what allows for double quotes to be omitted. Good point on iterating on the query! We already save the body into a temp file, so we can make Probe be able to run queries on a file. Adding it to the todo list. :) (In my own workflow I use gron a lot for getting an overview of the shape of unfamiliar JSON, super handy tool) As to an interactive shell… yes, 100%. Kicking ideas around something like that as well!
dj_mc_merlin 4y ago
Nice project. How are the request waterfalls done?
- hassy 4y ago
  thank you! we just grab the timings info for the request from the underlying HTTP library, and sprinkle some ASCII art on top. That part was inspired by httpstat [1] We want to extend those with support for Server-Timing next, and also Core Web Vitals [3] (via Playwright) for web pages. 1. https://github.com/reorx/httpstat https://github.com/reorx/httpstat 2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se... 3. https://web.dev/vitals/ https://web.dev/vitals/
ovebepari 4y ago
It would take me something extraordinary to move away from curl
- hassy 4y ago
  curl is amazing! I love curl. This is not meant to replace it completely. Probe just makes those most common use-cases friendlier, like looking at headers, or inspecting JSON retuned by an API with syntax highlighting, or querying without needing to reach for jq etc.
- blacksmith_tb 4y ago
  Sure, if you've got muscle memory for all the flags (plus it lets you reason about all the other programs that pipe stuff to curl). I personally find myself using httpie [1] at least as much. 1: https://httpie.io/ https://httpie.io/
  - deleted 4y ago
    [deleted]
  - ratorx 4y ago
    I think curlie is the middle ground and the best (IMO) in the space. A nicer CLI for interactive use and can use and output curl flags for scripting.
    - jicea 4y ago
      As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output. [1] https://hurl.dev https://hurl.dev
      - hassy 4y ago
        I haven’t come across Hurl before. It looks cool!
- 4y ago
arinlen 4y ago
Other than pretty printing the output, what does this offer that curl doesn't already do?
- hassy 4y ago
  A few things :) Syntax highlighting Request performance waterfalls Built-in JMESPath queries for JSON (same syntax as in AWS CLI). XML and HTML can be queried too. Ability to set checks on the response. This one is super handy for quick acceptance tests.
ushakov 4y ago
does this require Node.JS? edit: it looks like you need to install 1276 npm dependencies to use this tool https://github.com/artilleryio/artillery/network/dependencies https://github.com/artilleryio/artillery/network/dependencie... i'll pass, the risk is unacceptable for value it offers, especially in corporate environments
- Uptrenda 4y ago
  1276 deps... how is that even possible for a simple HTTP fetcher. Does node have no standard library? actually amazing.
- jsd1982 4y ago
  This is essentially what I was referring to with my comment below. If you can improve the installation experience such that I don't have to care that you developed with node.js, so much the better. If I were a node.js developer I surely wouldn't want my global package name space polluted like that. npm is not for end users.
  - ushakov 4y ago
    the solution is not write CLIs in JavaScript, unless you have a JS-library that you also want to make available as a CLI packaging Node in a binary won't solve the problem of JavaScript's plagued ecosystem and in fact i'd rather prefer to be able to mitigate the issues/update packages myself rather than waiting for a new binary to release
    - jsd1982 4y ago
      I agree with your conclusion to not write cli tools in Javascript. However, packaging node in a binary is pretty much what electron does today, albeit with a GUI. Regardless, it's not a problem I wish to spend any effort to solve since it's not my problem and I've (we've) already dismissed the premise (writing CLI tools in js) lol.
      - spoiler 4y ago
numlock86 4y ago
Nice idea. NodeJS and thousands of NPM dependencies are a big no-go with a red flag on top, though. One of the things something like Go probably would have been a better choice for. See "bombardier" for example.
- hassy 4y ago
  Bombardier is cool, but serves a very different use case. I mentioned elsewhere in the comments that we have a Docker image, and are working on other methods of installing the CLI to alleviate some of these dependency-related concerns. Getting side tracked here, but there seems to be a common sentiment when it comes to Node.js that it's uniquely insecure. Node.js has indeed had some unfortunate press when it comes to supply-chain security, but every other runtime is susceptible to those attacks (PiPy, Gems, Maven, Rust Crates). Ultimately of course, if you choose to avoid using any software built on top of those stacks, that's your choice. Artillery specifically is no different to any other Node.js-based project in how large the dependency tree is. VSCode for instance is used by millions of developers has 1.6k dependencies [1]. 1. https://github.com/microsoft/vscode/network/dependencies https://github.com/microsoft/vscode/network/dependencies
  - numlock86 4y ago
    > Bombardier is cool, but serves a very different use case. Yes, of course. I was just taking an arbitrary example from my common HTTP toolbox. > VSCode for instance is used by millions of developers has 1.6k dependencies [1]. VSCode is maintained by Microsoft plus a huge community and they are transparent about their process in regards of auditing/freezing dependencies.

110 points by hassy 4y ago

hassy 4y ago

clickable url: https://www.artillery.io/blog/swiss-army-knife-for-http-testing https://www.artillery.io/blog/swiss-army-knife-for-http-test...

jsd1982 4y ago

> npm install Sorry, lost me at this part. Have a better installation story like a single executable. Beyond that, what does this do that curl, wget, et al do not? How is your interface better than those? Is it robust in the face of errors? What http protocol versions does it support?

hassy 4y ago
if you’re on a Mac, Artillery has a Homebrew formula too. We have an official Docker image too. Other ways of installing are on the roadmap too (self contained tarballs and binaries). As to what it does better than curl/wget. Probe is geared towards interactive use with HTTP APIs. So you get syntax highlighting and pretty printing for JSON, built-in querying, request waterfall visualizations, and ability to set expectations on every response. HTTP/2 is supported.
- jsd1982 4y ago
  https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... Does artillery properly escape the environment variables into json strings? I don't see how it could if you're using bash expansion in double quotes. This feature would only work with string values without spaces or special characters. Don't really see any improvement here. https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... So by embedding a json query evaluator in your http client, this means that every time I iterate on my JMESPath query I'm making a new request? Is there an offline mode? What if my Json file is 200MiB? Or if the request was not idempotent? Just saying there's good reasons why json query features are not typically built into one-shot http request tools. However, if your tool was not one-shotand you had an interactive session and cached/stored the entire result and then allow the user to interactively query it, then I could see some better value here.
  - hassy 4y ago
    Yeah so the JSON quoting part is something I’m pretty pleased with. We use a YAML parser (JSON is a subset of YAML) to parse those values, which is what allows for double quotes to be omitted. Good point on iterating on the query! We already save the body into a temp file, so we can make Probe be able to run queries on a file. Adding it to the todo list. :) (In my own workflow I use gron a lot for getting an overview of the shape of unfamiliar JSON, super handy tool) As to an interactive shell… yes, 100%. Kicking ideas around something like that as well!

dj_mc_merlin 4y ago

Nice project. How are the request waterfalls done?

hassy 4y ago
thank you! we just grab the timings info for the request from the underlying HTTP library, and sprinkle some ASCII art on top. That part was inspired by httpstat [1] We want to extend those with support for Server-Timing next, and also Core Web Vitals [3] (via Playwright) for web pages. 1. https://github.com/reorx/httpstat https://github.com/reorx/httpstat 2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se... 3. https://web.dev/vitals/ https://web.dev/vitals/

ovebepari 4y ago

It would take me something extraordinary to move away from curl

hassy 4y ago
curl is amazing! I love curl. This is not meant to replace it completely. Probe just makes those most common use-cases friendlier, like looking at headers, or inspecting JSON retuned by an API with syntax highlighting, or querying without needing to reach for jq etc.
blacksmith_tb 4y ago
Sure, if you've got muscle memory for all the flags (plus it lets you reason about all the other programs that pipe stuff to curl). I personally find myself using httpie [1] at least as much. 1: https://httpie.io/ https://httpie.io/
- deleted 4y ago
  [deleted]
- ratorx 4y ago
  I think curlie is the middle ground and the best (IMO) in the space. A nicer CLI for interactive use and can use and output curl flags for scripting.
  - jicea 4y ago
    As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output. [1] https://hurl.dev https://hurl.dev
    - hassy 4y ago
      I haven’t come across Hurl before. It looks cool!
4y ago

arinlen 4y ago

Other than pretty printing the output, what does this offer that curl doesn't already do?

hassy 4y ago
A few things :) Syntax highlighting Request performance waterfalls Built-in JMESPath queries for JSON (same syntax as in AWS CLI). XML and HTML can be queried too. Ability to set checks on the response. This one is super handy for quick acceptance tests.

ushakov 4y ago

does this require Node.JS? edit: it looks like you need to install 1276 npm dependencies to use this tool https://github.com/artilleryio/artillery/network/dependencies https://github.com/artilleryio/artillery/network/dependencie... i'll pass, the risk is unacceptable for value it offers, especially in corporate environments

Uptrenda 4y ago
1276 deps... how is that even possible for a simple HTTP fetcher. Does node have no standard library? actually amazing.
jsd1982 4y ago
This is essentially what I was referring to with my comment below. If you can improve the installation experience such that I don't have to care that you developed with node.js, so much the better. If I were a node.js developer I surely wouldn't want my global package name space polluted like that. npm is not for end users.
- ushakov 4y ago
  the solution is not write CLIs in JavaScript, unless you have a JS-library that you also want to make available as a CLI packaging Node in a binary won't solve the problem of JavaScript's plagued ecosystem and in fact i'd rather prefer to be able to mitigate the issues/update packages myself rather than waiting for a new binary to release
  - jsd1982 4y ago
    I agree with your conclusion to not write cli tools in Javascript. However, packaging node in a binary is pretty much what electron does today, albeit with a GUI. Regardless, it's not a problem I wish to spend any effort to solve since it's not my problem and I've (we've) already dismissed the premise (writing CLI tools in js) lol.
    - spoiler 4y ago

numlock86 4y ago

Nice idea. NodeJS and thousands of NPM dependencies are a big no-go with a red flag on top, though. One of the things something like Go probably would have been a better choice for. See "bombardier" for example.

hassy 4y ago
Bombardier is cool, but serves a very different use case. I mentioned elsewhere in the comments that we have a Docker image, and are working on other methods of installing the CLI to alleviate some of these dependency-related concerns. Getting side tracked here, but there seems to be a common sentiment when it comes to Node.js that it's uniquely insecure. Node.js has indeed had some unfortunate press when it comes to supply-chain security, but every other runtime is susceptible to those attacks (PiPy, Gems, Maven, Rust Crates). Ultimately of course, if you choose to avoid using any software built on top of those stacks, that's your choice. Artillery specifically is no different to any other Node.js-based project in how large the dependency tree is. VSCode for instance is used by millions of developers has 1.6k dependencies [1]. 1. https://github.com/microsoft/vscode/network/dependencies https://github.com/microsoft/vscode/network/dependencies
- numlock86 4y ago
  > Bombardier is cool, but serves a very different use case. Yes, of course. I was just taking an arbitrary example from my common HTTP toolbox. > VSCode for instance is used by millions of developers has 1.6k dependencies [1]. VSCode is maintained by Microsoft plus a huge community and they are transparent about their process in regards of auditing/freezing dependencies.