5 ms·

> I tried using curl but out of the box it could not resume or retry the download. Maybe I'm misunderstanding, but curl has exactly that feature, it's the `-C`

by OskarS 3y ago

> I tried using curl but out of the box it could not resume or retry the download. Maybe I'm misunderstanding, but curl has exactly that feature, it's the `-C` flag. If you want retries, there's `--retry`. I find curls defaults pretty sane, personally, I wouldn't want either of those by default for a tool like curl.

deleted 3y ago
[deleted]
inopinatus 3y ago
Strong agree. The only misbehaviour I believe curl displays out of the box is globbing, which has burned me enough times that I’ve come to believe it would’ve been better disabled by default and enabled with -g instead of vice versa.
- grayhatter 3y ago
  do you have an example of the globbing that burns you?
vesinisa 3y ago
You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? Yes, I want retries. They should be the default for a user-facing tool. Try searching the curl's manual page for "retry". There are no less than 5 different interdependent flags for specifying retry behaviour: --retry-all-errors, --retry-connrefused, --retry-delay <seconds>, --retry-max-time <seconds> and --retry <num> If I "just" want it to retry, surely there's a boolean flag like "--retry" that enables sane defaults? Nope! --retry takes a mandatory integer argument of maximum number of retries. Surely I can set it to zero for a sane default? Nope again: "Setting the number to 0 makes curl do no retries." curl is a good tool if you know it through & through and want exact control over the transfer behaviour. I don't think it's a good tool if you just want to fetch a file and except your tool of choice to apply some sane behaviours for you to that end, that would probably make sense if you are a human rather than an application using a library.
- sherburt3 3y ago
  I thought that `-C -` checks the output file for the offset
  - madfucker69 3y ago
    Yes it does. How does the wget --continue work?
    - sherburt3 3y ago
      I don't know
- OskarS 3y ago
  > You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? But... it does, though. From the man page ( https://curl.se/docs/manpage.html#-C https://curl.se/docs/manpage.html#-C ) > Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out. I just tried it, works perfectly. I don't really see a difference between writing `--continue` in wget and `-C -` in curl. And the use case for specifying it is not so exotic, you might want to do range requests for all sorts of reasons. Look, it's fine if you prefer wget's command line: I don't think retrying a request is a reasonable default for a tool like curl, but reasonable people can disagree on that for sure. But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one.
  - vesinisa 3y ago
    No, `-C -` is not a flag. It is specifying the `-C` argument with obscure special value of `-`, which causes curl to determine the offset to continue from the output file length. This might be obvious to you if you are well-versed in curl command line, but it's by no means expected or obvious like a simple flag. > But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one. I've never claimed it doesn't. I've only demonstrated that the default options don't do it and enabling the behaviour is more difficult than it maybe should be for a simple tool. I fail to see the arrogance.

by OskarS 3y ago

deleted 3y ago

[deleted]

inopinatus 3y ago

Strong agree. The only misbehaviour I believe curl displays out of the box is globbing, which has burned me enough times that I’ve come to believe it would’ve been better disabled by default and enabled with -g instead of vice versa.

grayhatter 3y ago
do you have an example of the globbing that burns you?

vesinisa 3y ago

You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? Yes, I want retries. They should be the default for a user-facing tool. Try searching the curl's manual page for "retry". There are no less than 5 different interdependent flags for specifying retry behaviour: --retry-all-errors, --retry-connrefused, --retry-delay <seconds>, --retry-max-time <seconds> and --retry <num> If I "just" want it to retry, surely there's a boolean flag like "--retry" that enables sane defaults? Nope! --retry takes a mandatory integer argument of maximum number of retries. Surely I can set it to zero for a sane default? Nope again: "Setting the number to 0 makes curl do no retries." curl is a good tool if you know it through & through and want exact control over the transfer behaviour. I don't think it's a good tool if you just want to fetch a file and except your tool of choice to apply some sane behaviours for you to that end, that would probably make sense if you are a human rather than an application using a library.

sherburt3 3y ago
I thought that `-C -` checks the output file for the offset
- madfucker69 3y ago
  Yes it does. How does the wget --continue work?
  - sherburt3 3y ago
    I don't know
OskarS 3y ago
> You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? But... it does, though. From the man page ( https://curl.se/docs/manpage.html#-C https://curl.se/docs/manpage.html#-C ) > Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out. I just tried it, works perfectly. I don't really see a difference between writing `--continue` in wget and `-C -` in curl. And the use case for specifying it is not so exotic, you might want to do range requests for all sorts of reasons. Look, it's fine if you prefer wget's command line: I don't think retrying a request is a reasonable default for a tool like curl, but reasonable people can disagree on that for sure. But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one.
- vesinisa 3y ago
  No, `-C -` is not a flag. It is specifying the `-C` argument with obscure special value of `-`, which causes curl to determine the offset to continue from the output file length. This might be obvious to you if you are well-versed in curl command line, but it's by no means expected or obvious like a simple flag. > But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one. I've never claimed it doesn't. I've only demonstrated that the default options don't do it and enabling the behaviour is more difficult than it maybe should be for a simple tool. I fail to see the arrogance.