3 ms·

You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's ex

You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? Yes, I want retries. They should be the default for a user-facing tool. Try searching the curl's manual page for "retry". There are no less than 5 different interdependent flags for specifying retry behaviour: --retry-all-errors, --retry-connrefused, --retry-delay <seconds>, --retry-max-time <seconds> and --retry <num> If I "just" want it to retry, surely there's a boolean flag like "--retry" that enables sane defaults? Nope! --retry takes a mandatory integer argument of maximum number of retries. Surely I can set it to zero for a sane default? Nope again: "Setting the number to 0 makes curl do no retries." curl is a good tool if you know it through & through and want exact control over the transfer behaviour. I don't think it's a good tool if you just want to fetch a file and except your tool of choice to apply some sane behaviours for you to that end, that would probably make sense if you are a human rather than an application using a library.

sherburt3 3y ago
I thought that `-C -` checks the output file for the offset
- madfucker69 3y ago
  Yes it does. How does the wget --continue work?
  - sherburt3 3y ago
    I don't know
OskarS 3y ago
> You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? But... it does, though. From the man page ( https://curl.se/docs/manpage.html#-C https://curl.se/docs/manpage.html#-C ) > Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out. I just tried it, works perfectly. I don't really see a difference between writing `--continue` in wget and `-C -` in curl. And the use case for specifying it is not so exotic, you might want to do range requests for all sorts of reasons. Look, it's fine if you prefer wget's command line: I don't think retrying a request is a reasonable default for a tool like curl, but reasonable people can disagree on that for sure. But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one.
- vesinisa 3y ago
  No, `-C -` is not a flag. It is specifying the `-C` argument with obscure special value of `-`, which causes curl to determine the offset to continue from the output file length. This might be obvious to you if you are well-versed in curl command line, but it's by no means expected or obvious like a simple flag. > But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one. I've never claimed it doesn't. I've only demonstrated that the default options don't do it and enabling the behaviour is more difficult than it maybe should be for a simple tool. I fail to see the arrogance.
  - OskarS 3y ago
    > I've never claimed it doesn't. Yes you did: > You must specify the offset from where it should continue No, you mustn't, you can specify - and it does exactly what you want. The docs are very clear and even provide examples. At some point you should stop blaming curl for your inability to read a man page and admit that you were simply mistaken.
    - deleted 3y ago
      [deleted]
    - vesinisa 3y ago
      You still fail to understand that curl's -C does not behave as a simple flag but as an switch with a mandatory argument. And there's a magic special value for that argument that finally enables the expected behavior. It's unintuitive, hard to remember and not in agreement with usability. While I agree that curl is powerful I will not concede that it's CLI is user friendly.
      - grayhatter 3y ago
        It's not hard to remember if you're familiar with unixs tools and syntax. But no one is demanding you concede anything. The point of the conversation is explain the difference in expectations between how you expect a command line tool to act and how most people expect a command line tool to act. If I try to `cp src dest` and it fails. I don't want the tool to guess how to fix the issue, that's not it's job. Ditto for `dd` it shouldn't try to guess offsets. curl exists as a knife, you're expecting it to do the job have a food processor and blender combination. You're not wrong to expect a tool to behave that way you're wrong to expect curl to behave that way. And no one is asking you to concede that a knife is easier than a blender when you want a blender. everyone is pointing out it's not a blender it's a knife but you can still do everything you want to do.
semi 3y ago
What is a sane default for retries? is it to loop indefinitely? should it retry against the same TCP connection or establish a new one? To the same IP it picked the first time or a different one, or reresolve the DNS entirely? against the same resolver? IMO theres too much complexity for 'sane defaults' to not just be 'surprising behavior' for someone else's use case.
- vesinisa 3y ago
  It's indeed miraculous how Wget got this right
smaudet 3y ago
Not only are you wrong, I had the opposite experience - whether was all well and good for downloading pages, but it fell over for making e.g. authenticated POST requests. I think that might be possible now/have gotten better, but wget has always been too restrained for me. I am in agreement that wget has "sane" defaults i.e. it acts like a bot or web crawler, or basic browser. Curl has always been easier to get things done with, though. At least in the land of http requests.

sherburt3 3y ago

I thought that `-C -` checks the output file for the offset

madfucker69 3y ago
Yes it does. How does the wget --continue work?
- sherburt3 3y ago
  I don't know

OskarS 3y ago

> You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases? But... it does, though. From the man page ( https://curl.se/docs/manpage.html#-C https://curl.se/docs/manpage.html#-C ) > Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out. I just tried it, works perfectly. I don't really see a difference between writing `--continue` in wget and `-C -` in curl. And the use case for specifying it is not so exotic, you might want to do range requests for all sorts of reasons. Look, it's fine if you prefer wget's command line: I don't think retrying a request is a reasonable default for a tool like curl, but reasonable people can disagree on that for sure. But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one.

vesinisa 3y ago
No, `-C -` is not a flag. It is specifying the `-C` argument with obscure special value of `-`, which causes curl to determine the offset to continue from the output file length. This might be obvious to you if you are well-versed in curl command line, but it's by no means expected or obvious like a simple flag. > But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one. I've never claimed it doesn't. I've only demonstrated that the default options don't do it and enabling the behaviour is more difficult than it maybe should be for a simple tool. I fail to see the arrogance.
- OskarS 3y ago
  > I've never claimed it doesn't. Yes you did: > You must specify the offset from where it should continue No, you mustn't, you can specify - and it does exactly what you want. The docs are very clear and even provide examples. At some point you should stop blaming curl for your inability to read a man page and admit that you were simply mistaken.
  - deleted 3y ago
    [deleted]
  - vesinisa 3y ago
    You still fail to understand that curl's -C does not behave as a simple flag but as an switch with a mandatory argument. And there's a magic special value for that argument that finally enables the expected behavior. It's unintuitive, hard to remember and not in agreement with usability. While I agree that curl is powerful I will not concede that it's CLI is user friendly.
    - grayhatter 3y ago
      It's not hard to remember if you're familiar with unixs tools and syntax. But no one is demanding you concede anything. The point of the conversation is explain the difference in expectations between how you expect a command line tool to act and how most people expect a command line tool to act. If I try to `cp src dest` and it fails. I don't want the tool to guess how to fix the issue, that's not it's job. Ditto for `dd` it shouldn't try to guess offsets. curl exists as a knife, you're expecting it to do the job have a food processor and blender combination. You're not wrong to expect a tool to behave that way you're wrong to expect curl to behave that way. And no one is asking you to concede that a knife is easier than a blender when you want a blender. everyone is pointing out it's not a blender it's a knife but you can still do everything you want to do.

semi 3y ago

What is a sane default for retries? is it to loop indefinitely? should it retry against the same TCP connection or establish a new one? To the same IP it picked the first time or a different one, or reresolve the DNS entirely? against the same resolver? IMO theres too much complexity for 'sane defaults' to not just be 'surprising behavior' for someone else's use case.

vesinisa 3y ago
It's indeed miraculous how Wget got this right

smaudet 3y ago

Not only are you wrong, I had the opposite experience - whether was all well and good for downloading pages, but it fell over for making e.g. authenticated POST requests. I think that might be possible now/have gotten better, but wget has always been too restrained for me. I am in agreement that wget has "sane" defaults i.e. it acts like a bot or web crawler, or basic browser. Curl has always been easier to get things done with, though. At least in the land of http requests.