6 ms·

I appreciate it, but why not just: alias wcurl="curl -# -O $1" (I just tested this on MacOS).

by efitz 2y ago

nicce 2y ago
Curl does not handle reconnections or use caching like wget, for example, by default. And many other things which includes actually quite many arguments. You can see the list from this project.
- wakawaka28 2y ago
  Why not just use wget if you need all that? Wget does yet more useful things.
  - nicce 2y ago
    Then you would need two tools instead of one
    - wakawaka28 2y ago
      So? The tools are miniscule. Use the right tool for the right job. Shoehorning curl into the role of wget is silly.
      - bigiain 2y ago
        BUT BUT BUT - neither wget nor curl let me read my email! BRB, forking curl to add email client support. https://programmingphilosophy.org/posts/zawinskis-law/ https://programmingphilosophy.org/posts/zawinskis-law/
        gertop 2y ago
        Good news then, because curl supports IMAP!
      - nicce 2y ago
        > Shoehorning curl into the role of wget is silly. How is it silly if the identical functionality is there, but you just need to use more command-line arguments?
        wakawaka28 2y ago
        The functionality is not identical. Wget is much more high level. You should not need to be an HTTP and curl expert to download a file lol.
    - ocodo 2y ago
      I've been trying to tell people my hammer is perfect for cutting their hair, they won't listen. Until... I show them. They become unresponsive, but their hair is removed. Job done.
      - nicce 2y ago
        More comparable comparison here is that you have scissors which are more complex to use and will give better results when used correctly. But you are lazy and get new basic scissors.
        ocodo 2y ago
        Well no, I was suggesting "the golden hammer." ... where every problem is a nail.
  - dudus 2y ago
    MacOS comes with curl but no wget by default. Sure you can easily install it with homebrew. But it would be nice to have wget functionality by default.
anotherhue 2y ago
I felt the same but upon checking they do more than I expected. I like it.
guilherme-puida 2y ago
wcurl does a lot more than that, though: parallel downloads, retries, following redirects (and other things, that are all written in the blog post). There's also some value in providing a ready-to-use wrapper for users that install the curl package. Maybe wcurl will also show up in other distributions after a while, especially since Daniel also likes the idea.
- est 2y ago
  > parallel downloads, retries, following redirects There's aria2
deleted 2y ago
[deleted]
emmelaich 2y ago
You often want redirection too, so -L for --location.
- hackernudes 2y ago
  Totally agree! That one has burned me a lot. I generally use 'curl -LO' now instead of wget.
CJefferson 2y ago
The alias is closer to: alias wcurl = curl -# -O $1 --location --remote-name --remote-time --retry 10 --retry-max-time 10 --continue-at -" the main reason I use wget is because it automatically retries downloads, which is vital for my not-so-great internet, and an option I wish was on in curl by default, as half the world uses curl, and then keep trying to automatically download 400MB files, which I have trouble finishing. Then, if you look at the script, it's basically that, with some minor cleaning up and nicer error handling.
- jmprspret 2y ago
  FWIW, I think you can add some of those options to .curlrc if you wanted them to be added every time you invoke curl.
  - out-of-ideas 2y ago
    correct; .curlrc for the win! and you can use `-q` to have it exclude the curlrc and use only args you pass to it. curl has such an amazing amount of power, but also means it has a lot of options to utilize a lot of things folks take for granted (exactly like retries, ect)
    - lloeki 2y ago
      Makes me think that probably a lot of unattended shell scripts out there should use -q in case someone has a .curlrc altering the behaviour of curl, and thus breaking the expectations of the script. Maybe even should be a shellcheck rule.
      - out-of-ideas 2y ago
        same can be said for wget with --no-config (and really any app that runs on any system) - if you're in the business of automating it, those config-features need to not be ignored (or down right overwritten to your scripts desire) in docker-containers though, its more safe to assume no pre-loaded rc files are embedded (but 100% want to check your source-container for things like that) - but for running in some users workspace, the need to be careful is real. most of the time though, those options should be safe; only really need to not have auto-retries are when trying to get super accurate date (like observing odd behavior, you dont want thing to simply "auto-work" when finetuning something, or triaging an upstream-server or dns issue) i often write my scripts execing calls like `\curl -q ...` such that i get no user's alias of curl and ensures a consistant no-config amongst my system and others (although gnu curl vs mac curl (and other binaries of gnu vs mac vs busybox) are always the real fun part). (if the user has their own bash-function of curl then its on them for knowing their system will be special compared to others and results maybe inconsistent)
vietvu 2y ago
I felt weird reading the titles too, because maybe 2-3 years ago I have downloaded with `curl`, so why this? And also, I am pretty sure `wget` can do it and better too.
samueloph 2y ago
As others pointed out, you can do that, you can also set them in .curlrc, or you can write a script if you want to allow for multiple URLs to be downloaded in parallel (not possible with an alias), or now you can just use wcurl :) Note: wcurl sets a bit more flags than that, it also encodes the whitespaces in the URL and does parallel downloading of multiple URLs.
- deleted 2y ago
  [deleted]
- tssva 2y ago
  “or now you can just use wcurl :)” Sadly I can’t since it is dependent upon the util-linux version of getopt which means it fails on bsd and macOS systems. Understandable since it is always available on the specific target the script was written for and it does make life easier.
  - samueloph 2y ago
    argh, we are looking into fixing that now. I knew getopt was linux-specific but I thoughts the only impact was that the long form argument (--opt) would not work. I turns out it doesn't run at all instead. We should be able to fix this within the next few days, thank you!
    - samueloph 2y ago
      > We should be able to fix this within the next few days, thank you! It's fixed now, should work in non-linux environments.

I appreciate it, but why not just: alias wcurl="curl -# -O $1" (I just tested this on MacOS).

by efitz 2y ago

I appreciate it, but why not just: alias wcurl="curl -# -O $1" (I just tested this on MacOS).

Curl does not handle reconnections or use caching like wget, for example, by default. And many other things which includes actually quite many arguments. You can see the list from this project.

wakawaka28 2y ago
Why not just use wget if you need all that? Wget does yet more useful things.
- nicce 2y ago
  Then you would need two tools instead of one
  - wakawaka28 2y ago
    So? The tools are miniscule. Use the right tool for the right job. Shoehorning curl into the role of wget is silly.
    - bigiain 2y ago
      BUT BUT BUT - neither wget nor curl let me read my email! BRB, forking curl to add email client support. https://programmingphilosophy.org/posts/zawinskis-law/ https://programmingphilosophy.org/posts/zawinskis-law/
      - gertop 2y ago
        Good news then, because curl supports IMAP!
    - nicce 2y ago
      > Shoehorning curl into the role of wget is silly. How is it silly if the identical functionality is there, but you just need to use more command-line arguments?
      - wakawaka28 2y ago
        The functionality is not identical. Wget is much more high level. You should not need to be an HTTP and curl expert to download a file lol.
  - ocodo 2y ago
    I've been trying to tell people my hammer is perfect for cutting their hair, they won't listen. Until... I show them. They become unresponsive, but their hair is removed. Job done.
    - nicce 2y ago
      More comparable comparison here is that you have scissors which are more complex to use and will give better results when used correctly. But you are lazy and get new basic scissors.
      - ocodo 2y ago
        Well no, I was suggesting "the golden hammer." ... where every problem is a nail.
- dudus 2y ago
  MacOS comes with curl but no wget by default. Sure you can easily install it with homebrew. But it would be nice to have wget functionality by default.

anotherhue 2y ago

I felt the same but upon checking they do more than I expected. I like it.

guilherme-puida 2y ago

wcurl does a lot more than that, though: parallel downloads, retries, following redirects (and other things, that are all written in the blog post). There's also some value in providing a ready-to-use wrapper for users that install the curl package. Maybe wcurl will also show up in other distributions after a while, especially since Daniel also likes the idea.

est 2y ago
> parallel downloads, retries, following redirects There's aria2

deleted 2y ago

[deleted]

emmelaich 2y ago

You often want redirection too, so -L for --location.

hackernudes 2y ago
Totally agree! That one has burned me a lot. I generally use 'curl -LO' now instead of wget.

CJefferson 2y ago

The alias is closer to: alias wcurl = curl -# -O $1 --location --remote-name --remote-time --retry 10 --retry-max-time 10 --continue-at -" the main reason I use wget is because it automatically retries downloads, which is vital for my not-so-great internet, and an option I wish was on in curl by default, as half the world uses curl, and then keep trying to automatically download 400MB files, which I have trouble finishing. Then, if you look at the script, it's basically that, with some minor cleaning up and nicer error handling.

jmprspret 2y ago
FWIW, I think you can add some of those options to .curlrc if you wanted them to be added every time you invoke curl.
- out-of-ideas 2y ago
  correct; .curlrc for the win! and you can use `-q` to have it exclude the curlrc and use only args you pass to it. curl has such an amazing amount of power, but also means it has a lot of options to utilize a lot of things folks take for granted (exactly like retries, ect)
  - lloeki 2y ago
    Makes me think that probably a lot of unattended shell scripts out there should use -q in case someone has a .curlrc altering the behaviour of curl, and thus breaking the expectations of the script. Maybe even should be a shellcheck rule.
    - out-of-ideas 2y ago
      same can be said for wget with --no-config (and really any app that runs on any system) - if you're in the business of automating it, those config-features need to not be ignored (or down right overwritten to your scripts desire) in docker-containers though, its more safe to assume no pre-loaded rc files are embedded (but 100% want to check your source-container for things like that) - but for running in some users workspace, the need to be careful is real. most of the time though, those options should be safe; only really need to not have auto-retries are when trying to get super accurate date (like observing odd behavior, you dont want thing to simply "auto-work" when finetuning something, or triaging an upstream-server or dns issue) i often write my scripts execing calls like `\curl -q ...` such that i get no user's alias of curl and ensures a consistant no-config amongst my system and others (although gnu curl vs mac curl (and other binaries of gnu vs mac vs busybox) are always the real fun part). (if the user has their own bash-function of curl then its on them for knowing their system will be special compared to others and results maybe inconsistent)

vietvu 2y ago

I felt weird reading the titles too, because maybe 2-3 years ago I have downloaded with `curl`, so why this? And also, I am pretty sure `wget` can do it and better too.

samueloph 2y ago

As others pointed out, you can do that, you can also set them in .curlrc, or you can write a script if you want to allow for multiple URLs to be downloaded in parallel (not possible with an alias), or now you can just use wcurl :) Note: wcurl sets a bit more flags than that, it also encodes the whitespaces in the URL and does parallel downloading of multiple URLs.

deleted 2y ago
[deleted]
tssva 2y ago
“or now you can just use wcurl :)” Sadly I can’t since it is dependent upon the util-linux version of getopt which means it fails on bsd and macOS systems. Understandable since it is always available on the specific target the script was written for and it does make life easier.
- samueloph 2y ago
  argh, we are looking into fixing that now. I knew getopt was linux-specific but I thoughts the only impact was that the long form argument (--opt) would not work. I turns out it doesn't run at all instead. We should be able to fix this within the next few days, thank you!
  - samueloph 2y ago
    > We should be able to fix this within the next few days, thank you! It's fixed now, should work in non-linux environments.