3 ms·

I'm pretty sure wget won't buffer an entire document before writing anything through the pipe to sh. And since it's possible for retries to fail, you have to a

I'm pretty sure wget won't buffer an entire document before writing anything through the pipe to sh. And since it's possible for retries to fail, you have to assume sh could be sent a partial document, followed by EOF, no matter if http/https were used, or chunked/content-length, etc. It is surprising to me that the shell will honor a command line without a newline[1], but it does: $ (echo -n "echo rm -rf /tmp/mydir"; exit 1) | sh OUTPUT: rm -rf /tmp/mydir Obviously if tmpdir got truncated to /tmp, or even /, bad things would happen. [1] Not so suprising when you consider sh -c "command" works without a trailing newline, I suppose.

cbsmith 13y ago
> I'm pretty sure wget won't buffer an entire document before writing anything through the pipe to sh. And since it's possible for retries to fail, you have to assume sh could be sent a partial document, followed by EOF, no matter if http/https were used, or chunked/content-length, etc. Nope. By default wget will literally try forever (you can even shut down your network stack, wait for 10 minutes, and then turn it back on, and wget will proceed from there), and until it closes the stream, the shell interpreter will act as though data is still coming. Any normal way of telling wget to stop will also stop the shell interpreter. Worst case is you end up in some "in the middle" state of the script, but you won't be misinterpreting the script due to early termination of the stream.
cbsmith 13y ago
And it's not at all surprising that it honours a command without a newline. Commands are terminated by semicolons, newlines OR EOF. That's pretty much how all shells work. You really wouldn't want the last line of a shell script to be ignored simply because someone didn't put a newline at the end of it.
cbsmith 13y ago
And actually, wget will write to stdout with full buffering, which means that small files will actually be completely buffered before writing anything out to sh. That doesn't matter though because the problem that matters is NOT the buffering. Sure that can mean scripts execute partially and then exit, but they will never do something that wasn't intended. The problem the blog is talking about is when you get an EOF in the middle of a script.

cbsmith 13y ago

> I'm pretty sure wget won't buffer an entire document before writing anything through the pipe to sh. And since it's possible for retries to fail, you have to assume sh could be sent a partial document, followed by EOF, no matter if http/https were used, or chunked/content-length, etc. Nope. By default wget will literally try forever (you can even shut down your network stack, wait for 10 minutes, and then turn it back on, and wget will proceed from there), and until it closes the stream, the shell interpreter will act as though data is still coming. Any normal way of telling wget to stop will also stop the shell interpreter. Worst case is you end up in some "in the middle" state of the script, but you won't be misinterpreting the script due to early termination of the stream.

cbsmith 13y ago

And it's not at all surprising that it honours a command without a newline. Commands are terminated by semicolons, newlines OR EOF. That's pretty much how all shells work. You really wouldn't want the last line of a shell script to be ignored simply because someone didn't put a newline at the end of it.

cbsmith 13y ago

And actually, wget will write to stdout with full buffering, which means that small files will actually be completely buffered before writing anything out to sh. That doesn't matter though because the problem that matters is NOT the buffering. Sure that can mean scripts execute partially and then exit, but they will never do something that wasn't intended. The problem the blog is talking about is when you get an EOF in the middle of a script.