8 ms·
For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its comma
by JonathonW 11y ago
For my usage:
* Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't.
* Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line tool is more unix-y by default (outputs to stdout) and it's more flexible in terms of options. It's also present by default on more systems-- of note, OSX ships curl but not wget out of the box. Its backing library (libcurl) is also pretty nifty, but not really relevant to this comparison.
This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy: wget and curl do different things well and there's no reason why both shouldn't coexist in one's workflow.
- eru 11y agoLooks like a frontend to curl that has wget's defaults would be useful.
- theoh 11y agoMerge the code, hard link the filenames to the same executable and get it to change behavior based on the name it's invoked under.
- kcbanner 11y agoSounds like a lot of complexity for no benefit.
- eru 11y agoNo need to merge code. Take curl as is, and set different defaults based on executable name.
- deleted 11y ago[deleted]
- lucb1e 11y agoIn that case I'd love for it to include resuming downloads though, but given the differences I don't think merging is going to be more useful than just adding the feature in curl. Edit: nevermind, I just learned that wget can download a page's resources or even inline them (-k apparently), that's a bit of a way off from curl's purpose. Better keep them separate tools, although wget might benefit from using libcurl so they don't have to implement lots of stuff like https or http2.
- eli 11y agoDoesn't curl -C continue an incomplete download? Regardless agree that wget is better for just downloading a file.
- toupeira 11y ago-C requires you to specify the byte offset, and you also need --append or >> so it's not quite the same.
- hmpc 11y agoActually no. From the manpage: Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.
- raverbashing 11y agoCorrect, but very unintuitive
- lmz 11y agoAlso doesn't work with the command line utility's retry option as it gets the size only once and does not update it when retrying.
- foota 11y agoStupid question, but how do things like this resume from where they left off? Wouldn't the server need to be cooperating in this? Is that build into HTTP?
- zo1 11y agoYou specify a byte "range" when requesting a file via HTTP from the server. Not all servers support this, but most do these days. #Edit. More info here: https://en.wikipedia.org/wiki/Byte_serving https://en.wikipedia.org/wiki/Byte_serving
- dsmithatx 11y agoThe article is not coming up for me. Perhaps it's ycombinated. Anyway I agree curl is more of a developer tool although using it to download files is not the first thing that comes to mind. I use it daily to identify caching issues and redirect issues. The -SLIXGET flags in particular are very useful for this.
- Taniwha 11y agoYeah I've been doing some libcurl development today .... Online docs have been unavailable all day ..
- awkward 11y agoHuh. I've always thought of them as the opposite - wget is the full featured spidering tool, curl is the easy to run one when I need a command line thing or to bang web stuff into a janky copy and paste workflow.
- Legogris 11y agoThis is interesting. Do you want to give your reasons or is it just an arbitrary habit?
- awkward 11y agowget leaves files everywhere, and isn't useful without picking a few flags to use, carefully. I use it as a tool to check builds for broken links curl is a unix-way type program that interacts with standard out in a pretty predictable way.
- manyxcxi 11y ago> This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy: wget and curl do different things well and there's no reason why both shouldn't coexist in one's workflow. Totally agree. I love curl for testing API request/responses manually. It's usually a huge part of navigating my way around a new API that doesn't have a client library available for whatever language I'm using at that time. I also use it for weird requests that need special headers or authentication. Wget is the first thing I turn to when I'm downloading anything remote from the command line or scraping some remote content for analysis.
- mschuster91 11y agoUse the Chrome Postman extension together with Postman Interceptor. It's really awesome.
- hobarrera 11y agoI'd just like to mention HttpRequester here, which is a very similar FLOSS addon for Firefox.
- majewsky 11y agoRandom plug: Another invaluable tool for API analysis is http://mitmproxy.org http://mitmproxy.org.
- jsingleton 11y agoYes, wget is fantastic for mirroring www and ftp sites and I use it a lot for that purpose. It's magic [0]. I hadn't realized that it didn't support compression though, which might explain why it's so slow in some cases. Not normally a problem as it just runs in the background on a schedule. Curl supports gzip and deflate. It would be great if support for sdch and br were added too. Brotli is in Firefox 44 and can be enabled in Chrome Canary with a flag. SDCH has been in Chrome for a while and is used on Google servers. [0] http://www.litkicks.com/WgetMagic http://www.litkicks.com/WgetMagic
- girvo 11y agoInterestingly, I've somewhat replace my usual curl calls with HTTPie, at least on my mac; for distributed scripts it's still curl all the way.
- brobinson 11y agocurl is the one where I have to remember whether to use -o or -O when trying to download a file with the original filename and just use wget instead because it's faster than reading the curl man page.
- paulmd 11y agoI have the same problem. Curl is "unix-y" in the sense that the default options are optimized for a shell script and make no sense for interactive usage. bropages is where it's at for that kind of stuff. curl is actually their usage example :) http://bropages.org/ http://bropages.org/
- jandrese 11y agoTheir example will just dump the webpage to stdout! Almost certainly not the desired behavior given the comment. Then they include second example of a use case that almost nobody has, instead of giving the option that everyone is actually looking for.
- foota 11y agowow what the heck libcurl supports a ridiculous number of protocols (https://curl.haxx.se/libcurl/ https://curl.haxx.se/libcurl/)
- cathexis 11y agoPretty much. If I want to save a file: wget. If I want to do _anything_ else: curl. Yes you can write files with curl, no I don't use that functionality very often. I don't think of them as "end user" vs "developer" use cases so much as them being two great tools for different tasks. I do wish that -s was the curl default, since that stderr progress output is pretty lame.
- ucho 11y ago> Wget's [..] defaults are more sane Does it finally use filename from "Content-Disposition" without need for any switches?
- heinrich5991 11y agoThat sounds like a security issue, allowing the attacker to name a file like they want.
- Kurtz79 11y ago"This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy." Agreed, and even "emacs vs. vim" or "tabs vs. spaces" do not really need to be "emacs vs. vim" or "tabs vs. spaces"-type dichotomies.
- baldfat 11y agoI totally agree except over the years I have been using aria2 more and more instead of wget. aria2 supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink with the same sane wget syntax and defaults. example $ aria2c http://yourlink.com/file.* http://yourlink.com/file.* -x2 allows using 2 connections per host. https://aria2.github.io/ https://aria2.github.io/
- voltagex_ 11y agoI'm having to use aria2 more and more to maintain decent download speeds, occasionally even from Akamai and CloudFront. S3 seems to be the exception.