3 ms·
Wcurl Is Here
- Vecr 2y agoI tried it out, it works. Personally I'd write something like this in Rust not shell. It's not hard at all, you basically can just ignore the borrow checker because all the program is doing is spawning processes and a bit of text processing. The main issue with writing it in shell instead is incorrect error handling. I see a case that's a bit sketchy there. For now maybe don't expose wcurl to user/attacker controlled input.
- dgan 2y agoWhy would you bother with such a heavy language, just to wait on a single network event?
- shmerl 2y agoSwitched to wcurl from my own alias. It's neat. curl doesn't pollute $HOME unlike wget.
- tangus 2y agoHow does wget pollute $HOME?
- Izkata 2y ago~/.wget-hsts You can disable creating this file or change its location by adding a config option to ~/.wgetrc (or using args to wget every time) I think it's for disallowing http when an https connection has previously been established?
- Sohcahtoa82 2y agoIt's for HSTS: HTTP Strict Transport Security https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security So if you're wget-ing a file from a server that uses HTTPS and HSTS, but you only specify http:// http:// (or no protocol at all), then the next time you wget from that server, it will automatically change the URL to https:// https://
- mixmastamyk 2y agoIt's under .local in Fedora.
- shmerl 2y agoThey must have patched it, upstream wget doesn't do it.
- firesteelrain 2y agoThis is a great start! Some suggestions: - Allow insecure URLs by providing -k option automatically for https URLs - Auto follow HTTP redirects by providing -L option by default - Rather than use the curl-options setting, just accept —-user and —-pass as options then pass this to curl I know people will feel strongly about some of these however it would simplify curl usage for a majority of the download only use cases Other curl options can be used for the more “advanced” use cases like Headers
- viraptor 2y ago> Allow insecure URLs by providing -k option automatically for https URLs Why would you ever do that? This should be a very conscious choice if you decide to ignore the main system keeping the internet traffic trusted.
- firesteelrain 2y agoI use it all the time in house because we have in house, air gapped systems I know some people feel strongly about this one But the only time it could lead to a problem is if you pass user/pass and you have a MITM situation. So maybe only allow it if not passing user and pass If it’s just a download and we know we aren’t on a TOR node situation then privacy isn’t that great of a concern My two cents! Open to changing my mind
- saurik 2y agoA MITM situation is relevant even without a credential and isn't at all about privacy: an attacker can swap out a different file for the one you wanted to download.
- firesteelrain 2y agoYou are right. Hash (if provided) would still need to be verified upon download
- viraptor 2y ago
- Alifatisk 2y ago> remembering what curl options to use when they just want to download the contents of a URL is hard Which curl options do they mean? "curl -O URL"?
- brainzap 2y agocurl -LO url
- ilyagr 2y ago`curl -LOf url` is the simplest one I'm aware of that doesn't act too weirdly :) Without `-f`, it'll happily save the 404 error page if you do `curl -LO https://example.com/index.htmll https://example.com/index.htmll` After playing with `wcurl`, I may or may not remember that one. Also, wcurl puts a lot of effort into making `wcurl url1 url2` work equally predictably. Finally, wcurl doesn't yet support `wcurl https://example.com https://example.com` (without a filename at the end of the URL), but it might eventually.
- resoluteteeth 2y agoThe readme at https://github.com/curl/wcurl https://github.com/curl/wcurl explains some other stuff it does by default which makes sense if the goal is to download files: > By default, wcurl will: > Encode whitespaces in URLs; > Download multiple URLs in parallel if the installed curl's version is >= 7.66.0; > Follow redirects; > Automatically choose a filename as output; > Avoid overwriting files if the installed curl's version is >= 7.83.0 (--no-clobber); > Perform retries; > Set the downloaded file timestamp to the value provided by the server, if available; > Disable curl's URL globbing parser so {} and [] characters in URLs are not treated specially.
- ziml77 2y agoIs curl able to retry from the middle of a file? Beyond the sane defaults that's where wget has been nice for me; if the connection drops it can pick right back up where it left off.
- ChrisArchitect 2y ago[dupe] misleading Wcurl was here.....2 months ago. Discussion: https://news.ycombinator.com/item?id=40869458 https://news.ycombinator.com/item?id=40869458
- moreati 2y agoWas fully expecting wcurl to be an alternative name for curl on Windows, because Microsoft made "curl" an alias for their own http downloader command in Powershell.
- Izkata 2y agoI was thinking "web curl", like, an alternative to fetch() or XMLHttpRequest that exposed even more stuff.
- ndegruchy 2y ago`curl.exe` is actually the real deal curl, now. Though it's got less built options than a standard Linux distro version. I don't know if they still alias it in PowerShell, but you can still use curl.exe to get it.
- aaplok 2y ago> This is one often repeated reason why some users reach for wget instead of curl on the command line. Is there a reason not to do that? I've always used wget and curl interchangeably, because both meet my moderate needs. Is there a reason I should avoid wget?
- m463 2y agosometimes when using containers, curl was already installed and I use that. Other times it is wget. I might want to skip adding "apt-get install wget" etc
- lolpanda 2y agothe only reason i use wget over curl is that when i'm downloading a file, i don't have to specify the output file name with wget lol
- threePointFive 2y ago-O (for name based on end of the URL) or -J (for name based on the returned HTTP header)
- kelsey98765431 2y agoand wget is -o ; when i have a head full of tasks and code i only remember that they are different and tend to wget things, unless of course i am on a bsd and reach for fetch
- deleted 2y ago[deleted]
- jabr 2y ago"HTTPie CLI — modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more." https://github.com/httpie/cli https://github.com/httpie/cli
- alt187 2y agoI'll consider HTTPie when they make man pages I can look at in my CLI. This web "documentation" is atrocious.
- jkbr 2y agoWhat is it that makes the web documentation atrocious in your view? As for man pages, HTTPie has been shipping them since v3.2 [1]. Try `man http` or `http --manual`. [1] https://httpie.io/blog/httpie-3.2.0#man-pages https://httpie.io/blog/httpie-3.2.0#man-pages
- mixmastamyk 2y agoI preferred this for years but beware of new management, telemetry and little mention of it. This is from the desktop app but I've seen Little/Open Snitch warn about the CLI as well: https://github.com/httpie/desktop/issues/176 https://github.com/httpie/desktop/issues/176
- jkbr 2y agoHTTPie creator here. Just a clarification: HTTPie is still under the same management, and the CLI doesn’t have any telemetry. Little Snitch is probably warning you about requests to packages.httpie.io/latest.json [0]. That is done to let you know when a new version becomes available. It’s a static file hosted through GitHub pages [1] without any tracking. However, there’s currently a bug [2] that prevents the response from getting cached so that the request fires frequently for some users. It’s already fixed in a PR [3] and will be part of the upcoming release. [0] https://github.com/httpie/cli/blob/master/httpie/internal/update_warnings.py#L16 https://github.com/httpie/cli/blob/master/httpie/internal/up... [1] https://github.com/httpie/packages.httpie.io/blob/master/latest.json https://github.com/httpie/packages.httpie.io/blob/master/lat... [2] https://github.com/httpie/cli/issues/1527 https://github.com/httpie/cli/issues/1527 [3] https://github.com/httpie/cli/pull/1531 https://github.com/httpie/cli/pull/1531