8 ms·

Why do you need more performance from a CLI test tool? I'm honestly curious. It's basically curl (fast) plus simpler and easier interface and pretty printers.

Why do you need more performance from a CLI test tool? I'm honestly curious. It's basically curl (fast) plus simpler and easier interface and pretty printers. For performance in shell scripts you can use curl, and for troubleshooting the IO time dominates anyway.

deleted 6y ago
[deleted]
thallada 6y ago
Why use something slower when an equivalent faster tool is available? There's definitely a noticeable delay on my machine with starting up the python interpreter. Enough that it dominates most actual request times to fast servers. (`http get www.google.com` is ~460ms while `ht get www.google.com` is ~130ms) For a tool I'm constantly using to check APIs I'm developing, I really appreciate snappy commands that give me results that feel instantaneous.
pdimitar 6y ago
It adds up. I once tried to scrape an internal company website / API because we had no PDF exports and wanted to download + export to PDF, with `httpie`. It's an amazing tool but all the startup times compounded pretty badly. I switched to `curl` (which is a bit more pain until I got the full command line right, granted) and the script finished in ~95 minutes as opposed to the ~202 minutes it took with `httpie`. With about 0.5s startup time, it means every 120 requests add a full minute to the final time. And I had to scrape ~125K URLs back then. For a daily casual flow 0.5s startup time might not be much (although people like myself get irritated by that as well but I do recognize it's a minor inconvenience). But when doing mass-scripting such delays can very easily compound to non-trivial time inefficiencies.
- Scarbutt 6y ago
  To be honest, you approached this problem pretty badly.
  - michaelmrose 6y ago
    This might be a useful comment if you had spent the time to offer thoughts on how it could be better. As it is this comment serves only to gratify your ego whereas advice might help readers. It's worse than adding nothing at all.
  - pdimitar 6y ago
    It was supposed to be a one-off task and I was confident at the time that I can script it quickly. I still scripted it but it took at least half a day. I'll still behind the idea that one has to be able to do one-off tasks without starting a dedicated programming project. It's what scripting languages are meant for. But I did misjudge the speed at which I'll be able to finish the task, that much is true. Ironically I knew exactly what to do from the start but several small and annoying quirks of the shell scripting languages lost me quite a bit of time.
  - killingtime74 6y ago
    Your comment is approached pretty badly.
- dharmab 6y ago
  For mass-scripting I'd prefer to use a native HTTP library with connection pooling.
  - pdimitar 6y ago
    Sure, I can agree with that somewhat. Still, it was an one-off thing and I didn't want to turn it into a project. Using a for loop, `curl`, `wkhtmltopdf` and `parallel` eventually did the job.
    - j0057 6y ago
      I for one am totally on board with writing atrocious grep/sed/awk pipeline mashups, doing whatever it takes to get a one-off job done with as little programming as possible. Sometimes the important parameter to optimize is brain cycles spent, or time to result, not CPU time or optimal usage of network resources.
      - pdimitar 6y ago
        Absolutely. I'd even argue that the brain cycles spent should be the first and most important metric for one-off tasks. That's why I recently asked people here about what to use if not shell scripting. I think I'll either use a less confusing scripting language that is almost like bash/zsh -- namely `rc` -- or I'll brush off my OCaml skills. That need was borne out of my frustration that I routinely spent non-trivial amounts of time on one-off scripts so at one point I got fed up and started looking for alternatives.
        dharmab 6y ago
        aiohttp for Python is relatively performant and in a fairly easy to write language.
        pdimitar 6y ago
        Isn't that a library and not a CLI tool?
- inshadows 6y ago
  Why don't you use cURL then? cURL is written in C, which sure is teh performance you're looking for.
  - pdimitar 6y ago
    I did just that in the end. It took me some fiddling with headers and form parameters and file uploads, and it worked. It's just that httpie's (and thus ht's) CLI usage is a bit more ergonomic.
- mekster 6y ago
  I think people need to start distinguishing between CLI and TUI as TUI aren't really meant for scripting but CLI tools are.
  - pdimitar 6y ago
    Be that as it may, I still liked `httpie` (and `ht`) more than `curl` even though `curl` seems superior in terms of configurability.
bschwindHN 6y ago
Because slow startup time is obnoxious for any command-line script. There's no reason to start up an entire scripting virtual machine just to make an HTTP request. No one should be writing serious CLI tools in an interpreted language.
- inshadows 6y ago
  And yet for decades serious CLI tools are being written in POSIX shell, Bash, Zsh, Awk, Tcl, Perl, Ruby, Python, and no one really complained about speed until now, when hardware is so fast, finally people are starting to notice slow startup times??? C'mon. Python is not Java.
  - bschwindHN 6y ago
    I can definitely tell a difference, and it's irritating. The authors of these tools might not notice or care but I actively avoid them. Magic wormhole is one such example.

deleted 6y ago

[deleted]

thallada 6y ago

Why use something slower when an equivalent faster tool is available? There's definitely a noticeable delay on my machine with starting up the python interpreter. Enough that it dominates most actual request times to fast servers. (`http get www.google.com` is ~460ms while `ht get www.google.com` is ~130ms) For a tool I'm constantly using to check APIs I'm developing, I really appreciate snappy commands that give me results that feel instantaneous.

pdimitar 6y ago

It adds up. I once tried to scrape an internal company website / API because we had no PDF exports and wanted to download + export to PDF, with `httpie`. It's an amazing tool but all the startup times compounded pretty badly. I switched to `curl` (which is a bit more pain until I got the full command line right, granted) and the script finished in ~95 minutes as opposed to the ~202 minutes it took with `httpie`. With about 0.5s startup time, it means every 120 requests add a full minute to the final time. And I had to scrape ~125K URLs back then. For a daily casual flow 0.5s startup time might not be much (although people like myself get irritated by that as well but I do recognize it's a minor inconvenience). But when doing mass-scripting such delays can very easily compound to non-trivial time inefficiencies.

Scarbutt 6y ago
To be honest, you approached this problem pretty badly.
- michaelmrose 6y ago
  This might be a useful comment if you had spent the time to offer thoughts on how it could be better. As it is this comment serves only to gratify your ego whereas advice might help readers. It's worse than adding nothing at all.
- pdimitar 6y ago
  It was supposed to be a one-off task and I was confident at the time that I can script it quickly. I still scripted it but it took at least half a day. I'll still behind the idea that one has to be able to do one-off tasks without starting a dedicated programming project. It's what scripting languages are meant for. But I did misjudge the speed at which I'll be able to finish the task, that much is true. Ironically I knew exactly what to do from the start but several small and annoying quirks of the shell scripting languages lost me quite a bit of time.
- killingtime74 6y ago
  Your comment is approached pretty badly.
dharmab 6y ago
For mass-scripting I'd prefer to use a native HTTP library with connection pooling.
- pdimitar 6y ago
  Sure, I can agree with that somewhat. Still, it was an one-off thing and I didn't want to turn it into a project. Using a for loop, `curl`, `wkhtmltopdf` and `parallel` eventually did the job.
  - j0057 6y ago
    I for one am totally on board with writing atrocious grep/sed/awk pipeline mashups, doing whatever it takes to get a one-off job done with as little programming as possible. Sometimes the important parameter to optimize is brain cycles spent, or time to result, not CPU time or optimal usage of network resources.
    - pdimitar 6y ago
      Absolutely. I'd even argue that the brain cycles spent should be the first and most important metric for one-off tasks. That's why I recently asked people here about what to use if not shell scripting. I think I'll either use a less confusing scripting language that is almost like bash/zsh -- namely `rc` -- or I'll brush off my OCaml skills. That need was borne out of my frustration that I routinely spent non-trivial amounts of time on one-off scripts so at one point I got fed up and started looking for alternatives.
      - dharmab 6y ago
        aiohttp for Python is relatively performant and in a fairly easy to write language.
        pdimitar 6y ago
        Isn't that a library and not a CLI tool?
inshadows 6y ago
Why don't you use cURL then? cURL is written in C, which sure is teh performance you're looking for.
- pdimitar 6y ago
  I did just that in the end. It took me some fiddling with headers and form parameters and file uploads, and it worked. It's just that httpie's (and thus ht's) CLI usage is a bit more ergonomic.
mekster 6y ago
I think people need to start distinguishing between CLI and TUI as TUI aren't really meant for scripting but CLI tools are.
- pdimitar 6y ago
  Be that as it may, I still liked `httpie` (and `ht`) more than `curl` even though `curl` seems superior in terms of configurability.

bschwindHN 6y ago

Because slow startup time is obnoxious for any command-line script. There's no reason to start up an entire scripting virtual machine just to make an HTTP request. No one should be writing serious CLI tools in an interpreted language.

inshadows 6y ago
And yet for decades serious CLI tools are being written in POSIX shell, Bash, Zsh, Awk, Tcl, Perl, Ruby, Python, and no one really complained about speed until now, when hardware is so fast, finally people are starting to notice slow startup times??? C'mon. Python is not Java.
- bschwindHN 6y ago
  I can definitely tell a difference, and it's irritating. The authors of these tools might not notice or care but I actively avoid them. Magic wormhole is one such example.