6 ms·

All scripts should be run with "-eu". This prevents unbound variables from ever being used and will end the script on a failed command. If you want commands in

All scripts should be run with "-eu". This prevents unbound variables from ever being used and will end the script on a failed command. If you want commands in the script to fail, just drop the "e" and run with "-u". At my last job we put "-eu" in literally all of our scripts to avoid this very problem.

vog 10y ago
> All scripts should be run with "-eu". More specifically, every shell script should have "set -eu" at its second line, directly after the sh'bang line: #!... set -eu For debugging or logging purposes, sometimes adding -v and/or -x comes handy: #!... set -euvx
- shawabawa3 10y ago
  You can also do #!/bin/sh -eu Although i'm not sure if there are any advantages/disadvantages
  - wahern 10y ago
    Disadvantage is that nobody can use an alternative shell to run your script as intended. (Alternative here could mean a less buggy version of whatever is the system shell.)
- zwp 10y ago
  > should have "set -eu" at its second line Do you have a strong reason to prefer "set" over shebang flags? I have a slight preference for shebang flags so I can deliberately override them from the command line (but it's not a hill I'd die on): $ cat foo #!/bin/bash -eu cd /nowhere echo 'still here' $ ./foo ./foo: line 2: cd: nowhere: No such file or directory $ bash -c ./foo # thinking about system(3) ./foo: line 2: cd: nowhere: No such file or directory $ bash +e ./foo # override ./foo: line 2: cd: nowhere: No such file or directory still here $ EDIT: Google's shell style guide has "Executables must start with #!/bin/bash and a minimum number of flags. Use set to set shell options so that calling your script as bash <script_name> does not break its functionality." https://google.github.io/styleguide/shell.xml?showone=Which_Shell_to_Use#Which_Shell_to_Use https://google.github.io/styleguide/shell.xml?showone=Which_...
  - vog 10y ago
    Bash is not always /bin/bash. On some systems (BSDs) it is /usr/bin/bash. So the portable sh'bang line for bash is: #!/usr/bin/env bash And there you can't add "-eu" anymore, because only one argument is possible in sh'bang lines. (/usr/bin/env would try to find an executable named "bash -eu") So #!/usr/bin/env bash set -eu is the more portable approach. However, for "/bin/sh" this should work portably: #!/bin/sh -eu
    - saghm 10y ago
      > Bash is not always /bin/bash. On some systems (BSDs) it is /usr/bin/bash. Nitpick: on every BSD system I've used, it's /usr/local/bin/bash. But I definitely agree that using /usr/bin/env bash is the preferred solution for when you want to use bash (although I don't think it's even installed by default on most BSD's); as an Arch user (where /usr/bin/python is symlinked to python3 rather than python2), I appreciate those who go out of their way to use proper, portable shebangs.
- djsumdog 10y ago
  Is there a way to do this in Fish shell? So far the only thing I could find were bugs closed as duplicates that led to this open bug: https://github.com/fish-shell/fish-shell/issues/510 https://github.com/fish-shell/fish-shell/issues/510
leni536 10y ago
Agree, but be aware of subshells, these options don't propagate there. $ set -eu $ echo "$(false)" $
- teaearlgraycold 10y ago
  #!/bin/bash set -eu export SHELLOPTS
  - harryjo 10y ago
    That doesn't solve the problem that parent incorrectly explained. echo "$(false)" swallows the exit code from $(false). $ cat /tmp/test.sh set -euvx export SHELLOPTS echo "$(false)" echo "echo of string with failed subcommand does not kill script" $(false) echo "but consuming exit code does" $ /tmp/test.sh export SHELLOPTS + export SHELLOPTS echo "$(false)" false ++ false + echo '' echo "echo of string with failed subcommand does not kill script" + echo 'echo of string with failed subcommand does not kill script' echo of string with failed subcommand does not kill script $(false) false ++ false
- Slackwise 10y ago
  These are the kinds of pitfalls, aside from everything else in this thread, that make me just use Ruby for scripting instead. I mean, I can always just backtick pipe anything that would be convenient but avoid this nonsense.
wolfgang42 10y ago
To expound on this: "set -e" is short for "set -o errexit", that is, abort the script if a command returns with a non-zero exit code. "set -u" is short for "set -o nounset", that is, abort the script if a variable name is dereferenced when the variable hasn't been set. I also strongly suggest reading http://www.davidpashley.com/articles/writing-robust-shell-scripts/ http://www.davidpashley.com/articles/writing-robust-shell-sc... for more things that you should be aware of when writing bash scripts.
vkjv 10y ago
I usually add `pipefail` as well. set -euo pipefail
- jcassee 10y ago
  pipefail is bash-specific, no?
IgorPartola 10y ago
No no no. Clearly sh and bash are just broken. We need to write scripts in Rust or Haskell to avoid safety issues. </sarcasm>
- coldtea 10y ago
  Well, Rust and Haskell might not be for the job, but sh and bash are clearly broken and in many ways. -eu should have been the default from the start, for one.
  - TorKlingberg 10y ago
    Well, for scripts. set -e on an interactive shell is a bit of a challenge.
    - harryjo 10y ago
      interactive commands should abort the command on unset variable
      - tomjakubowski 10y ago
        "Abort command if variables are unset" is the -u flag, not -e. Both were mentioned long ago in this thread, but GP was talking about -e being painful for interactive shells. Imagine an interactive shell exiting as soon as a user's command failed!
        dllthomas 10y ago
        > Imagine an interactive shell exiting as soon as a user's command failed! Hardcore mode.
    - coldtea 10y ago
      A shell could, and I'm going out on a limb here, check whether it's interactive or not and just enforce the -e in the appropriate case...
GauntletWizard 10y ago
The other one worth recommending: -o pipefail; Which makes: false | true count as an error, where otherwise it would be ignored.
saurik 10y ago
I also do this. However, literally just hours ago, Greg Wooledge, author of The Bash Guide, posted the following to the GNU bug-bash mailing list. > Errexit (a.k.a. set -e) is horrible, and you should not be using it in any new shell scripts you write. It exists solely for support of legacy scripts. http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00170.html http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00170.... edit: I asked after this (for whatever reason, my email isn't yet showing in the archives; maybe it will appear later), and got an interesting response! http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00171.html http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00171....
wahern 10y ago
I also use set -f, which disables pathname expansion. It's trivial to enable pathname expansion around the few lines where that's desirable. Also -C so you don't accidentally clobber existing files. Useful to prevent symlink attacks, in addition to preventing stupid stuff. My standard non-interactive shell script preamble begins with set -e # strict error set -u # don't expand unbound variable set -f # disable pathname expansion set -C # noclobber
cellularmitosis 10y ago
I start out like that, but often I end up having to remove -u if I'm e.g. trying to check if $1 is populated. Is there a way to do that while still using -u?
- mdaniel 10y ago
  Same as any potentially unbound variable, by providing a (possibly empty) default value: if [ "${1:-}" == "my awesome value" ]; then ...

vog 10y ago

> All scripts should be run with "-eu". More specifically, every shell script should have "set -eu" at its second line, directly after the sh'bang line: #!... set -eu For debugging or logging purposes, sometimes adding -v and/or -x comes handy: #!... set -euvx

shawabawa3 10y ago
You can also do #!/bin/sh -eu Although i'm not sure if there are any advantages/disadvantages
- wahern 10y ago
  Disadvantage is that nobody can use an alternative shell to run your script as intended. (Alternative here could mean a less buggy version of whatever is the system shell.)
zwp 10y ago
> should have "set -eu" at its second line Do you have a strong reason to prefer "set" over shebang flags? I have a slight preference for shebang flags so I can deliberately override them from the command line (but it's not a hill I'd die on): $ cat foo #!/bin/bash -eu cd /nowhere echo 'still here' $ ./foo ./foo: line 2: cd: nowhere: No such file or directory $ bash -c ./foo # thinking about system(3) ./foo: line 2: cd: nowhere: No such file or directory $ bash +e ./foo # override ./foo: line 2: cd: nowhere: No such file or directory still here $ EDIT: Google's shell style guide has "Executables must start with #!/bin/bash and a minimum number of flags. Use set to set shell options so that calling your script as bash <script_name> does not break its functionality." https://google.github.io/styleguide/shell.xml?showone=Which_Shell_to_Use#Which_Shell_to_Use https://google.github.io/styleguide/shell.xml?showone=Which_...
- vog 10y ago
  Bash is not always /bin/bash. On some systems (BSDs) it is /usr/bin/bash. So the portable sh'bang line for bash is: #!/usr/bin/env bash And there you can't add "-eu" anymore, because only one argument is possible in sh'bang lines. (/usr/bin/env would try to find an executable named "bash -eu") So #!/usr/bin/env bash set -eu is the more portable approach. However, for "/bin/sh" this should work portably: #!/bin/sh -eu
  - saghm 10y ago
    > Bash is not always /bin/bash. On some systems (BSDs) it is /usr/bin/bash. Nitpick: on every BSD system I've used, it's /usr/local/bin/bash. But I definitely agree that using /usr/bin/env bash is the preferred solution for when you want to use bash (although I don't think it's even installed by default on most BSD's); as an Arch user (where /usr/bin/python is symlinked to python3 rather than python2), I appreciate those who go out of their way to use proper, portable shebangs.
djsumdog 10y ago
Is there a way to do this in Fish shell? So far the only thing I could find were bugs closed as duplicates that led to this open bug: https://github.com/fish-shell/fish-shell/issues/510 https://github.com/fish-shell/fish-shell/issues/510

leni536 10y ago

Agree, but be aware of subshells, these options don't propagate there. $ set -eu $ echo "$(false)" $

teaearlgraycold 10y ago
#!/bin/bash set -eu export SHELLOPTS
- harryjo 10y ago
  That doesn't solve the problem that parent incorrectly explained. echo "$(false)" swallows the exit code from $(false). $ cat /tmp/test.sh set -euvx export SHELLOPTS echo "$(false)" echo "echo of string with failed subcommand does not kill script" $(false) echo "but consuming exit code does" $ /tmp/test.sh export SHELLOPTS + export SHELLOPTS echo "$(false)" false ++ false + echo '' echo "echo of string with failed subcommand does not kill script" + echo 'echo of string with failed subcommand does not kill script' echo of string with failed subcommand does not kill script $(false) false ++ false
Slackwise 10y ago
These are the kinds of pitfalls, aside from everything else in this thread, that make me just use Ruby for scripting instead. I mean, I can always just backtick pipe anything that would be convenient but avoid this nonsense.

wolfgang42 10y ago

To expound on this: "set -e" is short for "set -o errexit", that is, abort the script if a command returns with a non-zero exit code. "set -u" is short for "set -o nounset", that is, abort the script if a variable name is dereferenced when the variable hasn't been set. I also strongly suggest reading http://www.davidpashley.com/articles/writing-robust-shell-scripts/ http://www.davidpashley.com/articles/writing-robust-shell-sc... for more things that you should be aware of when writing bash scripts.

vkjv 10y ago

I usually add `pipefail` as well. set -euo pipefail

jcassee 10y ago
pipefail is bash-specific, no?

IgorPartola 10y ago

No no no. Clearly sh and bash are just broken. We need to write scripts in Rust or Haskell to avoid safety issues. </sarcasm>

coldtea 10y ago
Well, Rust and Haskell might not be for the job, but sh and bash are clearly broken and in many ways. -eu should have been the default from the start, for one.
- TorKlingberg 10y ago
  Well, for scripts. set -e on an interactive shell is a bit of a challenge.
  - harryjo 10y ago
    interactive commands should abort the command on unset variable
    - tomjakubowski 10y ago
      "Abort command if variables are unset" is the -u flag, not -e. Both were mentioned long ago in this thread, but GP was talking about -e being painful for interactive shells. Imagine an interactive shell exiting as soon as a user's command failed!
      - dllthomas 10y ago
        > Imagine an interactive shell exiting as soon as a user's command failed! Hardcore mode.
  - coldtea 10y ago
    A shell could, and I'm going out on a limb here, check whether it's interactive or not and just enforce the -e in the appropriate case...

GauntletWizard 10y ago

The other one worth recommending: -o pipefail; Which makes: false | true count as an error, where otherwise it would be ignored.

saurik 10y ago

I also do this. However, literally just hours ago, Greg Wooledge, author of The Bash Guide, posted the following to the GNU bug-bash mailing list. > Errexit (a.k.a. set -e) is horrible, and you should not be using it in any new shell scripts you write. It exists solely for support of legacy scripts. http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00170.html http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00170.... edit: I asked after this (for whatever reason, my email isn't yet showing in the archives; maybe it will appear later), and got an interesting response! http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00171.html http://lists.gnu.org/archive/html/bug-bash/2017-03/msg00171....

wahern 10y ago

I also use set -f, which disables pathname expansion. It's trivial to enable pathname expansion around the few lines where that's desirable. Also -C so you don't accidentally clobber existing files. Useful to prevent symlink attacks, in addition to preventing stupid stuff. My standard non-interactive shell script preamble begins with set -e # strict error set -u # don't expand unbound variable set -f # disable pathname expansion set -C # noclobber

cellularmitosis 10y ago

I start out like that, but often I end up having to remove -u if I'm e.g. trying to check if $1 is populated. Is there a way to do that while still using -u?

mdaniel 10y ago
Same as any potentially unbound variable, by providing a (possibly empty) default value: if [ "${1:-}" == "my awesome value" ]; then ...