8 ms·

My take on the same topic: - use the unofficial strict mode: http://redsymbol.net/articles/unofficial-bash-strict-mode/ http://redsymbol.net/articles/unofficia

My take on the same topic: - use the unofficial strict mode: http://redsymbol.net/articles/unofficial-bash-strict-mode/ http://redsymbol.net/articles/unofficial-bash-strict-mode/ - use parameter substitutions like ${foo#prefix}, ${foo%suffix} instead of invoking sed/awk - process substitution instead of named pipes: <(), >() - know the difference between an inline group {} and a subshell () - use printf "%q" when passing variables to another shell (e.g. assembling a command locally and executing it via SSH)

lobster_johnson 9y ago
While process substitution is great on the surface, it comes with a troubling downside: There is no way, that I have found, to reliably catch errors. That is, this: some-command <(some-failing-command) ...will succeed. And -e and pipefail do nothing here. I have not found any way to push the error up. Plenty of questions on Stackoverflow and elsewhere, no good answers.
- mdaniel 9y ago
  pipefail do nothing here Isn't that because the syntax provided doesn't use a pipe? Which is why actually using a pipe, rather than paren redirection, is easier on the eyes and the shell. It reads more like the flow of text: maybe_failing() { if ! some-failing-command; then echo "That did not shake out" >&2 return 1 fi } maybe_failing | some-command Is that one of the things StackOverflow said? And if so, why is it not a "good" answer (verbosity concerns aside)?
  - lobster_johnson 9y ago
    This requires that some-command takes input from stdin (many commands don't) and that there's only a single input. What about: process-stuff --file <(make-input) \ --extra-data <(make-more-stuff | grep blah) This is the point at which one reaches for tempfiles, usually. The point remains that <() was made for a purpose, yet lacks error handling, thus undermining its usefulness to the point of uselessness. Edit: Isn't substitution using pipes, though? As in FIFO pipes? You get a file descriptor device which is closed at the end of the script. I don't know if the fd is wired directly to the command's stdout or whether the output is written in its entirety to a hidden tempfile first; the former sounds more natural and efficient.
    - tzahola 9y ago
      >I don't know if the fd is wired directly to the command's stdout or whether the output is written in its entirety to a hidden tempfile first; the former sounds more natural and efficient. Nope. <() and >() is equivalent to creating a named pipe with mkfifo and writing to it: tmp_pipe_dir="$(mktemp -d)" mkfifo "$tmp_pipe_dir/pipe" make-input >"$tmp_pipe_dir/pipe" & process-stuff --file "$tmp_pipe_dir/pipe" That's the whole point of it, so that the data can be consumed in parallel with the producer.
      - lobster_johnson 9y ago
        Right, that's what I meant. The command is run in a child process with its stdout writing to the pipe. The script gets the read end.
    - mdaniel 9y ago
      Yes, you absolutely have me about the stdin part But, the multiple case example highlights that if (for argument's sake) `make-input` and `make-more-stuff` had run _prior_ to that line, writing their (possibly empty) output to a (file|FIFO), how then would you want bash to behave? It would still open file descriptors to those (file|FIFO)s, which would still be just as blank. It seems to me that if one wishes more fine grained control over the error handling in a multi-subshell-fd-trickery situation, then creating the FIFO(s) and managing the sender's exit status is the supervising script's responsibility. a file descriptor device which is closed at the end of the script I checked, and it's actually not even at the end of the script; those fds only exist for that one child, as `process-stuff` is exec-ed by bash. whether the output is written in its entirety to a hidden tempfile It's not a temp-file, it's an actual file descriptor, which bash `dup2`s for the subprocess, then cheekily uses the `/dev/fd/63` syntax to make appear as a file; you can peer into its brain a little: $ showme() { echo "showme.args=$@" >&2 the_fd="${1##/dev/fd/}" cat <&${the_fd} } $ showme <(date -u) showme.args=/dev/fd/63 Sat Jan 6 21:31:25 UTC 2018 Thanks so much for highlighting this scenario; I learned a ton about how that works researching this answer. That's why I like answering stuff on S.O., too: win-win
oblio 9y ago
Parameter substitution is far from intuitive. Every time I have to open one of my older shell scripts (>3 months ago), I'm thankful I wrote comments. Otherwise I'd have to man/google things again. I really wish they'd use function names or something, instead (sub, etc.).
- tzahola 9y ago
  My "mnemonic" is that # means prefix, because every shell script starts with a shebang too. From this I can deduce that ## means longest prefix, therefore % and %% means suffix.

lobster_johnson 9y ago

While process substitution is great on the surface, it comes with a troubling downside: There is no way, that I have found, to reliably catch errors. That is, this: some-command <(some-failing-command) ...will succeed. And -e and pipefail do nothing here. I have not found any way to push the error up. Plenty of questions on Stackoverflow and elsewhere, no good answers.

mdaniel 9y ago
pipefail do nothing here Isn't that because the syntax provided doesn't use a pipe? Which is why actually using a pipe, rather than paren redirection, is easier on the eyes and the shell. It reads more like the flow of text: maybe_failing() { if ! some-failing-command; then echo "That did not shake out" >&2 return 1 fi } maybe_failing | some-command Is that one of the things StackOverflow said? And if so, why is it not a "good" answer (verbosity concerns aside)?
- lobster_johnson 9y ago
  This requires that some-command takes input from stdin (many commands don't) and that there's only a single input. What about: process-stuff --file <(make-input) \ --extra-data <(make-more-stuff | grep blah) This is the point at which one reaches for tempfiles, usually. The point remains that <() was made for a purpose, yet lacks error handling, thus undermining its usefulness to the point of uselessness. Edit: Isn't substitution using pipes, though? As in FIFO pipes? You get a file descriptor device which is closed at the end of the script. I don't know if the fd is wired directly to the command's stdout or whether the output is written in its entirety to a hidden tempfile first; the former sounds more natural and efficient.
  - tzahola 9y ago
    >I don't know if the fd is wired directly to the command's stdout or whether the output is written in its entirety to a hidden tempfile first; the former sounds more natural and efficient. Nope. <() and >() is equivalent to creating a named pipe with mkfifo and writing to it: tmp_pipe_dir="$(mktemp -d)" mkfifo "$tmp_pipe_dir/pipe" make-input >"$tmp_pipe_dir/pipe" & process-stuff --file "$tmp_pipe_dir/pipe" That's the whole point of it, so that the data can be consumed in parallel with the producer.
    - lobster_johnson 9y ago
      Right, that's what I meant. The command is run in a child process with its stdout writing to the pipe. The script gets the read end.
  - mdaniel 9y ago
    Yes, you absolutely have me about the stdin part But, the multiple case example highlights that if (for argument's sake) `make-input` and `make-more-stuff` had run _prior_ to that line, writing their (possibly empty) output to a (file|FIFO), how then would you want bash to behave? It would still open file descriptors to those (file|FIFO)s, which would still be just as blank. It seems to me that if one wishes more fine grained control over the error handling in a multi-subshell-fd-trickery situation, then creating the FIFO(s) and managing the sender's exit status is the supervising script's responsibility. a file descriptor device which is closed at the end of the script I checked, and it's actually not even at the end of the script; those fds only exist for that one child, as `process-stuff` is exec-ed by bash. whether the output is written in its entirety to a hidden tempfile It's not a temp-file, it's an actual file descriptor, which bash `dup2`s for the subprocess, then cheekily uses the `/dev/fd/63` syntax to make appear as a file; you can peer into its brain a little: $ showme() { echo "showme.args=$@" >&2 the_fd="${1##/dev/fd/}" cat <&${the_fd} } $ showme <(date -u) showme.args=/dev/fd/63 Sat Jan 6 21:31:25 UTC 2018 Thanks so much for highlighting this scenario; I learned a ton about how that works researching this answer. That's why I like answering stuff on S.O., too: win-win

oblio 9y ago

Parameter substitution is far from intuitive. Every time I have to open one of my older shell scripts (>3 months ago), I'm thankful I wrote comments. Otherwise I'd have to man/google things again. I really wish they'd use function names or something, instead (sub, etc.).

tzahola 9y ago
My "mnemonic" is that # means prefix, because every shell script starts with a shebang too. From this I can deduce that ## means longest prefix, therefore % and %% means suffix.