10 ms·
Show HN: Ht – HTTPie Clone in Rust
- johnhenry 6y agoCompiled vs interpreted -- is it faster?
- ducaale 6y agoYup, performance was one of the reasons I decided to port it to Rust. The other reason being that Rust gives you a single binary that is easier to deploy compared to python.
- inshadows 6y agoWhy do you need more performance from a CLI test tool? I'm honestly curious. It's basically curl (fast) plus simpler and easier interface and pretty printers. For performance in shell scripts you can use curl, and for troubleshooting the IO time dominates anyway.
- deleted 6y ago[deleted]
- thallada 6y agoWhy use something slower when an equivalent faster tool is available? There's definitely a noticeable delay on my machine with starting up the python interpreter. Enough that it dominates most actual request times to fast servers. (`http get www.google.com` is ~460ms while `ht get www.google.com` is ~130ms) For a tool I'm constantly using to check APIs I'm developing, I really appreciate snappy commands that give me results that feel instantaneous.
- pdimitar 6y agoIt adds up. I once tried to scrape an internal company website / API because we had no PDF exports and wanted to download + export to PDF, with `httpie`. It's an amazing tool but all the startup times compounded pretty badly. I switched to `curl` (which is a bit more pain until I got the full command line right, granted) and the script finished in ~95 minutes as opposed to the ~202 minutes it took with `httpie`. With about 0.5s startup time, it means every 120 requests add a full minute to the final time. And I had to scrape ~125K URLs back then. For a daily casual flow 0.5s startup time might not be much (although people like myself get irritated by that as well but I do recognize it's a minor inconvenience). But when doing mass-scripting such delays can very easily compound to non-trivial time inefficiencies.
- Scarbutt 6y agoTo be honest, you approached this problem pretty badly.
- michaelmrose 6y agoThis might be a useful comment if you had spent the time to offer thoughts on how it could be better. As it is this comment serves only to gratify your ego whereas advice might help readers. It's worse than adding nothing at all.
- pdimitar 6y agoIt was supposed to be a one-off task and I was confident at the time that I can script it quickly. I still scripted it but it took at least half a day. I'll still behind the idea that one has to be able to do one-off tasks without starting a dedicated programming project. It's what scripting languages are meant for. But I did misjudge the speed at which I'll be able to finish the task, that much is true. Ironically I knew exactly what to do from the start but several small and annoying quirks of the shell scripting languages lost me quite a bit of time.
- killingtime74 6y agoYour comment is approached pretty badly.
- dharmab 6y agoFor mass-scripting I'd prefer to use a native HTTP library with connection pooling.
- bschwindHN 6y agoBecause slow startup time is obnoxious for any command-line script. There's no reason to start up an entire scripting virtual machine just to make an HTTP request. No one should be writing serious CLI tools in an interpreted language.
- inshadows 6y agoAnd yet for decades serious CLI tools are being written in POSIX shell, Bash, Zsh, Awk, Tcl, Perl, Ruby, Python, and no one really complained about speed until now, when hardware is so fast, finally people are starting to notice slow startup times??? C'mon. Python is not Java.
- bschwindHN 6y agoI can definitely tell a difference, and it's irritating. The authors of these tools might not notice or care but I actively avoid them. Magic wormhole is one such example.
- pdimitar 6y agoNice work! As shared down-thread, I really loved and wanted to use `httpie` but the non-trivial amount of startup has put me off. Very happy that you made a Rust alternative because I really don't like `curl` that much -- it requires quite the amount of incantations for non-trivial requests. `ht` and `httpie` definitely improve ergonomics at important places. So, kudos!
- adamnemecek 6y agoHow could it not be?
- iruoy 6y agoIt definitely is. % hyperfine 'http get EXAMPLE.COM' 'ht get EXAMPLE.COM' -w 10 -r 100 Benchmark #1: http get EXAMPLE.COM Time (mean ± σ): 200.8 ms ± 7.4 ms [User: 178.9 ms, System: 21.1 ms] Range (min … max): 166.5 ms … 221.0 ms 100 runs Benchmark #2: ht get EXAMPLE.COM Time (mean ± σ): 22.7 ms ± 3.2 ms [User: 8.7 ms, System: 5.6 ms] Range (min … max): 15.6 ms … 31.3 ms 100 runs Summary 'ht get EXAMPLE.COM' ran 8.84 ± 1.27 times faster than 'http get EXAMPLE.COM'
- jdright 6y agoSorry about asking this, I don't mean to be mean or anything, please don't take offense on this - I'm honestly intrigued. I would like to understand from where a question like this comes, I understand different people have different experiences and backgrounds, but being on this site I would assume this would be kinda obvious. Can you share anything about your experience that can help me understand it?
- staticassertion 6y agoI know many people who wouldn't necessarily understand that compiled programs tend to be faster, because it just hasn't come up, or because in their domain it's not really important or even clear. If you've been working primarily with Javascript, for example, or even worse, Typescript, the question of "is this thing compiled or interpreted" is "yes". Compilation becomes a piece of either build tooling, where the artifact isn't a concrete binary, or even an attribute of runtime. It really wouldn't be too hard to imagine a shop built on Javascript and Python, where compilation just doesn't factor into the system in a meaningful way.
- jdright 6y agoThanks, I see what you mean. I think the TypeScript example makes a great point on the difference of perspectives.
- unrealhoang 6y ago> the question of "is this thing compiled or interpreted" is "yes" Isn’t that answer applied to every programming language?
- Volundr 6y agoFWIW as a developer whose worked in both interpreted and compiled languages (but not Python) it's obvious to me that the compiled version will be faster, but it's not obvious that it'd be meaningfully faster. I'd actually expect network latencies to dominate here in many cases. As another commenter mentioned I suspect what's being measured here is the Python startup time rather than performance once things are running. My guess would be a faster to start interpreter (like lua or v8) would rival ht performance, while a slower to start compiled language like the JVM would be as "slow" or slower than httpie.
- hlieberman 6y agoI think the biggest value of this vs httpie is that it doesn't use the system TLS libraries (openssl/gnutls/whatever) but instead uses rustls. Less attack surface, though admittedly if someone has a compromise that works on arbitrary HTTPS traffic, you're probably pretty boned anyway.
- arusahni 6y agoAnother advantage: your installation isn't tethered to a system Python implementation that may go away and nuke the env.
- ducktective 6y agopyenv/pipx is a thing... The binaries of Rust/Go replicas should also be updated but to my experience few package managers provide packages for apps written in those languages.
- frenchman99 6y agoSo now you need Python + pipenv + the actual program? Versus just the binary with go/rust (or any compiled language for that matter)? I think that's exactly the kind of complexity arusahni was referring to. Being able to deploy a single binary with `go get` or `cargo build` without worrying about anything else is really convenient. I guess it depends on the use case though, I use Python extensively at work and have little complaints.
- ducktective 6y agoMore like `pipx install httpie`. To update all python apps : `pipx upgrade-all`. For setting up pipx, you can use a pyenv managed python, like latest version, independent of whatever version is installed on system/distro. And you do this only one time. pipenv is for managing virtual envs not python itself. `go get` and `cargo build` don't "deply" the binary but merely build it. It's `sudo make install` vs `apt update` all over again. Point is, from the complexity of app management POV, those language don't magically solve anything that python doesn't, not to mention the bigger size of the binaries built with Rust/Go. I do choose rg/fd/fzf over python-equivalents all the time but I acknowledge that I have to manually keep track of installing/updating these programs.
- tyingq 6y agoNice work, saw this referenced on another Show HN recently: https://news.ycombinator.com/item?id=25939042 https://news.ycombinator.com/item?id=25939042 Adding HEAD as one of the methods would be welcome so we could get headers without pulling the whole page. Also, I did find the arguments for the -p switch in the source, but that should probably go in the README too.
- flitzofolov 6y agoThis is great! I've also come to really like curlie (https://curlie.io/ https://curlie.io/). It is written in in Go and provides an `httpie` like interface with `curl` like performance.
- ComputerGuru 6y agoCurlie is actually way better because it doesn’t mess with the default curl arguments. You don’t need to (re)learn httpie’s overly verbose command line parameters if you already know curl, and you don’t need to rewrite scripts or snippets that currently use curl.
- johanzm 6y agoThen why not just use curl?
- petre 6y agoNo pretty colors, no JSON formatting, some kind of security vulnerability every month. Maybe the last one isn't quite fair though given curl's usage and the number of security researchers looking at its code.
- pastage 6y agoCurl does output JSON nowdays, but I should try it for the colors.
- Phlogistique 6y agoCurlie uses curl, so security concerns about curl are certainly not a good reason to use Curlie.
- petre 6y agoThanks. I have previously used bat which is not just a frontend but it hasn't had commits in a while. https://github.com/astaxie/bat https://github.com/astaxie/bat
- saghm 6y agoI took a stab at something like this a few years back (https://github.com/saghm/rural https://github.com/saghm/rural), but yours is much fuller-featured. I'll definitely try this out soon!
- nitsky 6y agoThank you so much for doing this! I can feel the slow startup time of httpie every time I use it. ht has made it disappear! For users of Arch Linux, I submitted an AUR package: https://aur.archlinux.org/packages/ht-bin/ https://aur.archlinux.org/packages/ht-bin/.
- O_H_E 6y agoDamnit AUR, it really has got everything. *Mumbles ubuntu-based user"
- O_H_E 6y agoWait, fyi it is actually 404ing rn.
- psYchotic 6y agoFor any Arch user running into this thread, wanting to install this: the project went through a rename (to xh), and a package has been added to the Arch "community" repo. So all you need to install this now is: pacman -Syu xh
- yagizdegirmenci 6y agoThank you!
- monadic3 6y agoWhat is httpie? Edit: popular cli replacement for curl for basic http requests. See: https://httpie.io/ https://httpie.io/
- newscracker 6y agoI’m intrigued by the current version number and what seems to be a quick pace of development. I see that you have a “HTTPie feature parity checklist” (issue #4) on the issues list. Is that the sole indicator (aside from bugs) of what a version 1.0 would be like? Also, please post another Show HN when the backlog is almost done.
- ducaale 6y agoThanks, I will try post it one more time once the tool is stabilized. >I see that you have a “HTTPie feature parity checklist” (issue #4) on the issues list. Is that the sole indicator (aside from bugs) of what a version 1.0 would be like? Yes, that pretty much sums it. Also The list might expand if I notice other HTTPie features I forgot about.
- indemnity 6y agoNice work! These days I try to avoid interpreted CLI tools, you never know when they’re going to stop working. Rust/Go CLI tools basically work on Linux, Max, Windows without too much fuss. Wish someone would rewrite Homebrew in Rust!
- chme 6y agoI never looked into Homebrew as I know that its mainly for MacOS. Since you are sort of making the connection for Homebrew to work without fuss on Linux and Windows as well, if implemented in Rust, I am asking myself what it would offer that a user of Linux and Windows is missing? Ordinary Linux distributions already have a package management and there is scoop and chocolaty for Windows.
- codetrotter 6y agoI didn’t read their comment that way. Although Homebrew does also exist for Linux, I think the absolute majority use it on macOS. Personally, being a user of Homebrew on macOS I’ve been thinking about wanting brew to be rewritten in Rust too. My own reason being that I think it would be faster when run, compared to the version they have of it currently which can be quite slow sometimes. And because I like Rust as a language and for its memory safety features. So I’d rather see brew implemented in Rust than in say C or C++, if a compiled language was to be used.
- antidocker 6y agoNo. Homebrew should use C++ and C++ is beautiful than that bogus Rust. I'm willing to war with Rust Cultists. Billion dollars would be spent on Ads in coming weeks. I will start from YouTube to show how cool is C++
- Xylakant 6y agoI don’t think that homebrew will ever get rewritten in anything other than ruby. It may be feasible to rewrite brew itself, but all the formulae are written in ruby, so moving away from it would insta-delete the entire ecosystem.
- jicea 6y agoVery nice work, seems really fast! Since it’s written in Rust, do you plan to release a Windows version ? HTTPie requires Python so Ht could have another advantage being easier to install. We are also developing an HTTP client in Rust using curl (it’s an overwhelmed place!), it’s still very young but I link here anyway https://hurl.dev https://hurl.dev [edited] there is already a Windows version, so really good job! > The release page contains prebuilt binaries for Linux, macOS and Windows.
- maccam94 6y ago> HTTP client in Rust using curl Which could possibly use Hyper under the hood :-P https://github.com/curl/curl/wiki/Hyper https://github.com/curl/curl/wiki/Hyper
- clumsysmurf 6y ago> The release page contains prebuilt binaries for Linux, macOS and Windows. Thank you so, so much.
- dbrgn 6y agoVery nice! httpie is my favorite HTTP client, and it's always nice if command line tools run a bit faster. Unfortunately the name clashes with a binary that's part of TeXlive: $ pacman -Qo /usr/bin/ht /usr/bin/ht is owned by texlive-core 2020.57066-1 But I guess then the packagers will need to rename the "ht" binary to "http" (and add a conflict with httpie) when packaging.
- donaldihunter 6y agoReally nice. It needs a style that is suitable for a terminal with white background though.
- BiteCode_dev 6y agoSome httpie tricks: Use --offline to see the request but not perform it: http --offline get google.com GET / HTTP/1.1 Accept: application/json, */*;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive Content-Length: 5 Content-Type: application/json Host: google.com User-Agent: HTTPie/2.4.0 Pipe stuff to httpie to put it in the payload: echo "Hello HN" | http --offline get google.com GET / HTTP/1.1 Accept: application/json, */*;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive Content-Length: 13 Content-Type: application/json Host: google.com User-Agent: HTTPie/2.4.0 Hello HN And use http-prompt, because it's awesome: https://github.com/httpie/http-prompt https://github.com/httpie/http-prompt
- remram 6y agoHate to be that guy, but it's a bit weird to pick the HT part of HTTP, which stands for HyperText, when the tool only deals with the Transfer Protocol.