4 ms·

This won’t work if you retry another function and are relying on set -e to detect errors: set -e retry() { until "$@" do sleep 1 done

This won’t work if you retry another function and are relying on set -e to detect errors: set -e retry() { until "$@" do sleep 1 done } f() { false echo oh no } retry f # oh no set -e is ignored when a function executes in the context of a predicate. Shell scripts are great for when you just need to run stuff. If something bad happens you notice it and run another command to ameliorate the error. This is how most interactive shell sessions pan out. As soon as you need to handle errors programmatically I heartily suggest that you switch to a language with fewer quirks than sh.

inferiorhuman 4y ago
That quote from Jurassic Park comes to mind. As soon as you need to handle errors programmatically I heartily suggest that you switch to a language with fewer quirks than sh. Basically that. `set -e` is evil.
- 411111111111111 4y ago
  No, it's not. Nor did the parent indicate such. set -euo pipefail should be the default so that the script exits on at least some unexpected state/command results. If you want to do more then that with various fallbacks etc... then use another language.
  - inferiorhuman 4y ago
    The problem with set -e is that you end up failing in inconsistent and non-intuitive ways. e.g. https://stackoverflow.com/questions/4072984/how-do-i-get-the-effect-and-usefulness-of-set-e-inside-a-shell-function https://stackoverflow.com/questions/4072984/how-do-i-get-the... https://serverfault.com/questions/143445/what-does-set-e-do-and-why-might-it-be-considered-dangerous https://serverfault.com/questions/143445/what-does-set-e-do-... pipefail is worse because tools like grep return non-zero for things that aren't inherently errors. Error handling in sh-like shells is archaic enough that if you're reaching for it you should strongly consider reaching for a different language.
    - 411111111111111 4y ago
      you're missintepreting what i said. I'm saying that a script which would have the potential to fail unintuitively from `set -eup pipefail` probably shouldn't be written in shell and most definitely shouldn't continue to execute once an unexpected state has occured. You're right that there are a lot of cases in which non-zero exitcodes are the expected behaviour. But if you're accepting these commands with non-zero exitcodes then you're already doing error handling by verifying the output of the commands (at least hopefully), which was the original criteria from the OP for writing the script in another language.
- vore 4y ago
  I think your script chugging along in the face of a failure and running whatever commands that probably relied on what happened before it blindly is probably more evil, though.
  - inferiorhuman 4y ago
    set -e creates two problems. The first is that what is considered "failure" by the shell is not consistent or intuitive. The second is that what each of your tools consider failure is also not necessarily intuitive. You're getting a false sense of security at best. E.g. grep considers no matches worthy of a non-zero (a.k.a. error) exit code while df considers an invalid block size an error worthy of a exit code of zero. Ostensibly printf(1) exits with non-zero on an error, but a format specifier with insufficient arguments is not an error.
  - pmarreck 4y ago
    So basically, Go? ;)
pkkm 4y ago
You should never use set -e. Because shell scripts use exit status both to indicate failure and as a kind of a boolean, this option has to try to guess your intent by using a bunch of rules that are convoluted and can differ between shells, or even between versions. It's a matter of time before they do something you don't expect. Putting || exit or || return everywhere really is the better option if you can't switch to a better language like Python. I highly recommend https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls. It should be required reading before anyone writes a shell script.
- Too 4y ago
  No no no. Shell scripts without -e is one of the biggest footguns since computers were invented. Second being shell scripts with -e enabled ;) As others mentioned -u and pipefail are also highly recommended. Even mundane shit like ‘cd’ can fail where you least expect it, requiring || exit on practically every line of the script. Forget one place and boom. Defeating the premise of “it’s just a short “simple” script so let’s not write it in a sane language”. If you want to write serious scripts and spend weeks scrutinizing every line, NASA style, it is perhaps better to explicitly handle every potential error. But good luck doing that in a large team of junior web developers just glueing stuff together in the build pipeline. The pitfalls with -e linked above are mostly related to functions, if your script reached the size of requiring functions you have kind of lost already.
  - pkkm 4y ago
    > Even mundane shit like ‘cd’ can fail where you least expect it, requiring || exit on practically every line of the script. Forget one place and boom. This is true, but a much better solution is for everyone to use shellcheck. You can require that all shell scripts pass it without warnings and that every shellcheck disable comment has a good explanation attached; the first part can be automated with a pre-commit hook. In return, you get a script that is explicit and predictable rather than relying on a set of rules which you don't understand, and which may come back to bite you later when you change shells or even bash versions. > spend weeks scrutinizing every line, NASA style This is a colossal exaggeration. I've switched from set -e to putting || return or || exit everywhere and it barely takes more time after I got in the habit. On the rare occasions I get distracted and forget, shellcheck reminds me. I have my editor configured to run it on the fly and underline lines with warnings.

inferiorhuman 4y ago

That quote from Jurassic Park comes to mind. As soon as you need to handle errors programmatically I heartily suggest that you switch to a language with fewer quirks than sh. Basically that. `set -e` is evil.

411111111111111 4y ago
No, it's not. Nor did the parent indicate such. set -euo pipefail should be the default so that the script exits on at least some unexpected state/command results. If you want to do more then that with various fallbacks etc... then use another language.
- inferiorhuman 4y ago
  The problem with set -e is that you end up failing in inconsistent and non-intuitive ways. e.g. https://stackoverflow.com/questions/4072984/how-do-i-get-the-effect-and-usefulness-of-set-e-inside-a-shell-function https://stackoverflow.com/questions/4072984/how-do-i-get-the... https://serverfault.com/questions/143445/what-does-set-e-do-and-why-might-it-be-considered-dangerous https://serverfault.com/questions/143445/what-does-set-e-do-... pipefail is worse because tools like grep return non-zero for things that aren't inherently errors. Error handling in sh-like shells is archaic enough that if you're reaching for it you should strongly consider reaching for a different language.
  - 411111111111111 4y ago
    you're missintepreting what i said. I'm saying that a script which would have the potential to fail unintuitively from `set -eup pipefail` probably shouldn't be written in shell and most definitely shouldn't continue to execute once an unexpected state has occured. You're right that there are a lot of cases in which non-zero exitcodes are the expected behaviour. But if you're accepting these commands with non-zero exitcodes then you're already doing error handling by verifying the output of the commands (at least hopefully), which was the original criteria from the OP for writing the script in another language.
vore 4y ago
I think your script chugging along in the face of a failure and running whatever commands that probably relied on what happened before it blindly is probably more evil, though.
- inferiorhuman 4y ago
  set -e creates two problems. The first is that what is considered "failure" by the shell is not consistent or intuitive. The second is that what each of your tools consider failure is also not necessarily intuitive. You're getting a false sense of security at best. E.g. grep considers no matches worthy of a non-zero (a.k.a. error) exit code while df considers an invalid block size an error worthy of a exit code of zero. Ostensibly printf(1) exits with non-zero on an error, but a format specifier with insufficient arguments is not an error.
- pmarreck 4y ago
  So basically, Go? ;)

pkkm 4y ago

You should never use set -e. Because shell scripts use exit status both to indicate failure and as a kind of a boolean, this option has to try to guess your intent by using a bunch of rules that are convoluted and can differ between shells, or even between versions. It's a matter of time before they do something you don't expect. Putting || exit or || return everywhere really is the better option if you can't switch to a better language like Python. I highly recommend https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls. It should be required reading before anyone writes a shell script.

Too 4y ago
No no no. Shell scripts without -e is one of the biggest footguns since computers were invented. Second being shell scripts with -e enabled ;) As others mentioned -u and pipefail are also highly recommended. Even mundane shit like ‘cd’ can fail where you least expect it, requiring || exit on practically every line of the script. Forget one place and boom. Defeating the premise of “it’s just a short “simple” script so let’s not write it in a sane language”. If you want to write serious scripts and spend weeks scrutinizing every line, NASA style, it is perhaps better to explicitly handle every potential error. But good luck doing that in a large team of junior web developers just glueing stuff together in the build pipeline. The pitfalls with -e linked above are mostly related to functions, if your script reached the size of requiring functions you have kind of lost already.
- pkkm 4y ago
  > Even mundane shit like ‘cd’ can fail where you least expect it, requiring || exit on practically every line of the script. Forget one place and boom. This is true, but a much better solution is for everyone to use shellcheck. You can require that all shell scripts pass it without warnings and that every shellcheck disable comment has a good explanation attached; the first part can be automated with a pre-commit hook. In return, you get a script that is explicit and predictable rather than relying on a set of rules which you don't understand, and which may come back to bite you later when you change shells or even bash versions. > spend weeks scrutinizing every line, NASA style This is a colossal exaggeration. I've switched from set -e to putting || return or || exit everywhere and it barely takes more time after I got in the habit. On the rare occasions I get distracted and forget, shellcheck reminds me. I have my editor configured to run it on the fly and underline lines with warnings.