4 ms·

This adds `set -o pipefail` to POSIX sh, which causes a whole pipeline to fail (non-zero exit code) if one or more of the commands in the pipeline fail.

• throwaway984393 2y ago
  Sad. Use of that option is almost always a mistake. It only leads to undebuggable silent failures.
  • Joker_vD 2y ago
    I'd rather both have this option and have it work reliably. It's ridiculous that export VAR=$(cmd1 | cmd2) does not count as a pipefail when cmd1 or cmd2 fail but VAR=$(cmd1 | cmd2) does, so the "correct" way to set an environment variable from a pipeline's output is actually VAR=$(cmd1 | cmd2) export VAR
  • ykonstant 2y ago
    Pipefail is useful and very hard to emulate on pure POSIX; you need to create named fifos, break the pipeline into individual redirections and check for error on each line. And that is fine; but sometimes you want to treat a pipeline as a "single command" and then you can use pipefail to abort the pipeline on error. Then you can handle the error at the granularity of the entire pipeline without caring which part failed. Lastly, I am confused as to the "silent" failures; maybe you are thinking of combining this with `set -e`? Then yes, that is bad and I recommend against the combination; but then again, I and most advanced scripters recommend against shotgunning `set -e` in the first place. Use it in specific portions of the script when appropriate, and use proper error handling otherwise.
    • zelphirkalt 2y ago
      Why does `set -e` make a pipeline fail silently?
      • ykonstant 2y ago
        `set -e` makes the script abort and is often used in lieu of proper error handing: set -e command command [fails] command Whether the above reports error or not depends on the command; when you have a pipeline failing in the above way, it is even sneakier: set -e command command | command | command [fails] command You are reliant on all commands in the pipeline being verbose about failure to signal error. None of the above is advisable. The advisable code is error_handler() { proper error handling; } command || error_handler "parameter" command || error_handler "parameter" { command | command | command; } || error_handler "parameter" { set -e exceptional section that needs to be bailed out set +e } command || error_handler "parameter"
        • skydhash 2y ago
          Error handling like that makes sense if you’re writing a program. But if you just want a script for an automation, `set -e` is enough.
          • ykonstant 2y ago
            It is not; Greg's wiki further explains why, if the silent failure problem above is not enough reason.
            • Joker_vD 2y ago
              Gee, imagine if shells with errexit option enabled wrote some diagnostic output to stderr before exiting. "Add your own error checking instead", how do I check which piece of pipeline has failed, exactly? The PIPESTATUS variable is bash-specific and was not standardized.
              • ykonstant 2y ago
                ? Why are you replying to me? My position was pretty clear: "Pipefail is useful and very hard to emulate on pure POSIX; you need to create named fifos, break the pipeline into individual redirections and check for error on each line. And that is fine; but sometimes you want to treat a pipeline as a "single command" and then you can use pipefail to abort the pipeline on error. Then you can handle the error at the granularity of the entire pipeline without caring which part failed." By the way, I never script in Bash; I only script in POSIX primitives using dash as my executable.
• akdor1154 2y ago
  Holy balls that's like Christmas!
• rightbyte 2y ago
  Really? Wont that break piping grep?
  • WJW 2y ago
    Probably, so don't `set -o pipefail` in scripts that pipe into grep.
    • rightbyte 2y ago
      Ah ok I read it as 'sets it by default' for some reason.
• zelphirkalt 2y ago
  Does it? It is not mentioned anywhere in the post. Can you post a reference to your source?
  • noselasd 2y ago
    The post only have a few highlights. The Posix specs are only for paying IEEE customers though, but https://pubs.opengroup.org/onlinepubs/9799919799/ https://pubs.opengroup.org/onlinepubs/9799919799/ mentions it.
    • arp242 2y ago
      That is the POSIX spec, no? It's at: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V... (no permalink, search for "pipefail")
• deskr 2y ago
  If you're writing scripts, use that and don't forget -e and -u -e Exit immediately if a pipeline (which may consist of a single simple command), a list, or a compound command (see SHELL GRAMMAR above), exits with a non-zero status -u Treat unset variables and parameters other than the special parameters "@" and "*" as an error when performing parameter expansion
  • deleted 2y ago
    [deleted]
  • ykonstant 2y ago
    For `set -u` I mostly agree. For `set -e` see my comment below and Greg's wiki: http://mywiki.wooledge.org/BashFAQ/105 http://mywiki.wooledge.org/BashFAQ/105
    • deskr 2y ago
      > and they still fail to catch even some remarkably simple cases I totally agree. Although I'd say that there isn't anything "remarkably simple" about writing a bash script. Anything in the shell scripting world that seems remarkably simple is just because one hasn't realised the ghosts and horrors that lurk in the shadows. But I'll use -e anytime. It feels like having a protective proton pack at least.