4 ms·

Bash Strict Mode (2014)

48 points by dcminter 1y ago

deathanatos 1y ago
Or just don't use Bash. Python is a great scripting language, and won't blow your foot off if you try to iterate through an array. Other than that, yeah, if you must use bash, set -eu -o pipefail; the IFS is new and mildly interesting idea to me. > The idea is that if a reference is made at runtime to an undefined variable, bash has a syntax for declaring a default value, using the ":-" operator: Just note that defaulting an undefined variable to a value (let's use a default value of "fallback") for these examples is, ${foo-fallback} The syntax, ${foo:-fallback} means "use 'fallback' if foo is unset or is equal to "". (The :, specifically triggers this; there's a bunch of others, like +, which is "use alternate value", or, you'll get the value if the parameter is defined, nothing otherwise. if [[ "${foo+set}" == "set" ]]; then # foo is not undefined. fi And similarly, ${foo:+triggered} will emit triggered if foo is set and not empty.) See "Parameter Expansion" in the manual. I hate this syntax, but it is the syntax one must use to check for undefined-ness.
- xelxebar 1y ago
  Permit me to vent just a bit: > Python is a great scripting language, and won't blow your foot off if you try to iterate through an array. I kind of hate that every time the topic of shell scripting comes up, we get a troop of comments touting this mindless nonsense. Python has footguns, too. Heck, it's absolutely terrible and hacky if you try to do concatenative programming with it. Does that mean it should never be used? Instead of bashing the language, why not learn bash the language? IME, most of the industry has just absorbed shell programming haphazardly through osmosis, and almost always tries to shove the square pegs of OOP and FP into the round hole that is bash. No wonder people are having a bad time. In contrast, a data-first design that heavily normalizes data into line-oriented tables and passes information around in pipes results in simple, direct code IME. Stop trying to use arrays and embrace data normalization and text. Also, a lot of pain comes from simply not learning the facilities, e.g. the set builtin obviates most uses of string munging and exec: set -- "$@" --file 'filename with spaces.pdf' set -- "$@" 'data|blob with "dangerous" characters' set -- "$@" "$etc" some_command "$@" Anyway, the senseless bash hate is somewhat of a pet peeve of mine. Exunt.
  - 3eb7988a1663 1y ago
    All languages have foot guns, but bash is on the more explodey end of the scale. It is not senseless to note that if you can use a safer tool, you should consider it. C/C++ got us really far, but greenfield projects are moving to safer languages where they can. Expert low level programmers, armed with all of the available linting tools are still making unfortunate mistakes. At some point we should switch to something better.
    - oguz-ismail 1y ago
      > At some point we should switch to something better. Agree. Python isn't it though
    - xelxebar 1y ago
      In my years of reading and writing bash as well as Python for sysops tasks, I'd say that bash is the more reliable workhorse of the two. Python tends to encourage a kind of overengineering, resulting in more bugs overall. Many times I've seen hundreds of lines of Python or Typescript result from the attempt to replace just a few lines of bash! The senselessness I object to is not the conscientious choice of tooling or discussion of the failings thereof; it's the fact that every single bash article on here sees the same religious refrain, "Python is better than bash. Period." It's like if every article about vim saw a flood of comments claiming that vim is okay for light editing, but for any real programming we should use a real editor like emacs. If you open vim expecting emacs but with a few different bindings, then it might just explode in you face. If you use bash expecting to be able to program just like Python but with slightly different syntax, then it's not surprising to feel friction. IME, bash works exceptionally well using a data-oriented, text-first design to program architecture. It's just unfortunate that very little of the industry is even aware of this style of programming.
matheusmoreira 1y ago
This honestly should be the default for all scripts. There are so many little annoyances in bash that would make it great if they were changed and improved. Sadly there's just no changing certain things. My number one wishlist feature was a simple library system. Essentially just let me source files by name by searching in some standard user location. I actually wrote and submitted patches for this but it just didn't work out. Maintained my own version for a while and it was nice but not enough to justify the maintenance burden. Bash's number one feature is being the historical default shell on virtually every Linux distribution, without that there's no point. At least we've got shellcheck.
- dhamidi 1y ago
  The `source` builtin searches through all directories on PATH, not quite a library system, but useable with a bit of discipline in the to-be-sourced shellscripts.
  - matheusmoreira 1y ago
    Yeah. I sent patches that added a flag that made it search special library paths instead. source -l some-library
- abathur 1y ago
  When you say library system, do you mean something more or less like a separate search path and tools for managing it? I've written a little about how we can more or less accomplish something like meaningfully-reusable shell libraries in the nix ecosystem. I think https://www.t-ravis.com/post/shell/the_missing_comprehensive_package_manager_for_shell/ https://www.t-ravis.com/post/shell/the_missing_comprehensive... lays out the general idea but you can also pick through how my bashrc integrates libraries/modules (https://github.com/abathur/bashrc.nix https://github.com/abathur/bashrc.nix). (I'm just dropping these in bin, since that works with existing search path for source. Not ideal, but the generally explicit nature of dependencies in nix minimizes how much things can leak where they aren't meant to go.)
  - 1y ago
heybrendan 1y ago
From 2014 [1]. This seems to be posted once per year (at least); however, hardly a complaint as the discussion tends to be high quality. My personal mantra is if it's over 10~20 lines, I should arguably be using another language, like Python (and perhaps correspondingly, subprocess [2], if I'm in a hurry). [1] https://web.archive.org/web/20140523002853/http://redsymbol.net/articles/unofficial-bash-strict-mode/ https://web.archive.org/web/20140523002853/http://redsymbol.... [2] https://docs.python.org/3/library/subprocess.html https://docs.python.org/3/library/subprocess.html
- esafak 1y ago
  It's time to stop using these archaic shells, and make sure the newer ones are packaged for and included with popular operating systems. Better scripting languages have been offered in shells for a long time: https://en.wikipedia.org/wiki/Scsh https://en.wikipedia.org/wiki/Scsh These days I've settled on https://www.nushell.sh/ https://www.nushell.sh/
  - tux1968 1y ago
    I think https://oils.pub/ https://oils.pub/ has a good shot at being the eventual replacement because it has a very strong transition story. Being backward compatible, while allowing you to progressively embrace a modern replacement, is pretty powerful.
  - ZYbCRq22HbJ2y7 1y ago
    i don't think nushell is better than bash for day to day things. it is nice when you need it though, and then you can just run it, like with any shell.
    - esafak 1y ago
      Why would you do that when nushell is strictly better? It's bash that I drop into when I rarely need to, which is when someone gives a bash recipe, not nushell.
chubot 1y ago
I use essentially this, but I think this post is over 10 years old (needs a date), and it's now INCOMPLETE. bash introduced an option to respect rather than ignore errors within command sub processes years ago. So if you want to be safer, do something like: #!/bin/bash set -euo pipefail shopt -s inherit_errexit That works as-is in OSH, which is part of https://oils.pub/ https://oils.pub/ (edit: the last time this came up was a year ago, and here's a more concrete example - https://lobste.rs/s/1wohaz/posix_2024_changes#c_9oo1av https://lobste.rs/s/1wohaz/posix_2024_changes#c_9oo1av ) --- But that's STILL incomplete because POSIX mandates that errors be LOST. That is, it mandates broken error handling. For example, there what I call the "if myfunc" pitfall set -e my-deploy-func # errors respected if ! my-deploy-func; then # errors lost echo failed fi my-deploy-func || echo fail # errors lost But even if you fix that, it's still not enough. --- I describe all the problems in this doc, e.g. waiting for process subs: YSH Fixes Shell's Error Handling (errexit) - https://oils.pub/release/latest/doc/error-handling.html https://oils.pub/release/latest/doc/error-handling.html Summary: YSH fixes all shell error handling issues. This was surprisingly hard and required many iterations, but it has stood up to scrutiny. For contrast, here is a recent attempt at fixing bash, which is also incomplete, and I argue is a horrible language design: https://lobste.rs/s/kidktn/bash_patch_add_shopt_for_implicit_place https://lobste.rs/s/kidktn/bash_patch_add_shopt_for_implicit...
- aidenn0 1y ago
  What are your thoughts on pipefail in bash? I know ysh (and maybe osh too?) are smart about a -13 result from a pipe, but I stopped using pipefail since the return value can be based on a race, causing things to randomly not work at some point in the future. [edit] Per the link you posted osh treats a -13 result as success with sigpipe_status_ok, but I'm still interested in your thoughts on if pipefail is better or worse to recommend as "always on" for bash.
  - chubot 1y ago
    All my scripts use pipefail, and I haven't run into problems I think `head` tends to cause SIGPIPE in practice, and that is what sigpipe_status_ok is for but I stopped using pipefail since the return value can be based on a race, causing things to randomly not work at some point in the future. Hm I'd be interested in an example of this The order that processes in a pipeline finish is non-deterministic, but that should NOT affect the status, with either pipefail or not. That is, the status should be deterministic ... it's possible that SIGPIPE is an exception to that, though that wouldn't affect YSH.
kayson 1y ago
Or use shellcheck: https://www.shellcheck.net/ https://www.shellcheck.net/
- burnt-resistor 1y ago
  Tl;dr: Use both because they aren't mutex. Shellcheck isn't a complete solution, and running -e mode is essential to smaller bash files. Shellcheck even knows if a script is in -e mode or not.
cendyne 1y ago
If you ever try something like this with another bash-hackery tool like NVM, you will have a very bad time.
jacob2161 1y ago
Stylistically, I much prefer #!/bin/bash set -o errexit set -o nounset set -o pipefail It reminds me when I wrote a lot of Perl: #!/usr/bin/perl use strict; use warnings; I also prefer --long --args everywhere possible and a comment on any short flag unless it's incredibly common. I've been writing all my Bash scripts like this for ~15 years and it's definitely the way to go.
- Waterluvian 1y ago
  Yeah! I feel like long args very strongly fits with the overall idea of this blog post: when you are running a script you care about different things and want different behaviours than when you’re working interactively. The short args are great for when you’re typing into a terminal. But when you’re writing and saving a script, be descriptive! One of the most irritating things about bash (and Linux, I guess?) is how magical all the -FEU -lh 3 -B 1 incantations are. They give off a vibe I call “nerd snobbish” where it’s a sort of “oh you don’t know what that all means? How fretful!”
- burnt-resistor 1y ago
  Too damn verbose and you're assuming bash is at /bin. This will cause problems in nix and other environments where bash should be found on the path instead. Always, always unless the script is guaranteed to not be portable to other platforms: #!/usr/bin/env bash ...
  - jacob2161 1y ago
    You're assuming env is in /usr/bin? There are real environments where it's at /bin/env. Running a script with bash script.sh Works on any system with bash in the path, doesn't require that it be executable, and is usually the ideal way to execute them. I don't really believe in trying to write portable Bash scripts. There's a lot more to it than just getting the shebang right. Systems and commands tend to have subtle differences and flags are often incompatible. Lots of branching logic makes them hideous quickly. I prefer to write a script for each platform or just write a small portable Go program and compile it for each platform. I'm almost always writing Bash scripts for Linux that run in Debian-based containers, where using /bin/bash is entirely correct and safe. Regarding verbosity, most people come to appreciate a little verbosity in exchange for clarity with more experience. Clever and compact code is almost never what you want in production code.
vivzkestrel 1y ago
Again another great read about why you should NOT use this https://mywiki.wooledge.org/BashFAQ/105 https://mywiki.wooledge.org/BashFAQ/105
vivzkestrel 1y ago
Lots of arguments against not using set -euo pipefail https://www.reddit.com/r/commandline/comments/g1vsxk/comment/fniifmk/ https://www.reddit.com/r/commandline/comments/g1vsxk/comment... anything you wanna say about this?
- awestroke 1y ago
  euo pipefail has been the one good thing with bash. I'll start looking at alternatives now
- jacob2161 1y ago
  That post is quite nitpicky, pointing to edge cases and old version behavior. The essence of it is that writing reliable Bash scripts requires significant domain knowledge, which is true. Bash scripts are great if you've been doing it for a very long time or you're doing something simple and small (<50 LOC). If it's complicated or large, you should just write it in a proper programming language anyway.
  - heresie-dabord 1y ago
    > something simple and small (<50 LOC). If it's complicated or large, you should just write it in a proper programming language anyway. Regardless of LOC, eject from bash/awk as soon as you need a data structure, and choose a better language.
    - jacob2161 1y ago
      Yeah, that's a good point. It's less about LOC than complexity (data structures, complicated string parsing, handling binary data, etc). And that's actually the advice I follow myself. Some longish Bash scripts are very maintainable and reliable, because they're still quite regular and simple.
burnt-resistor 1y ago
set -Eeuo pipefail It's missing -E to respect traps.
tomhow 1y ago
Use the unofficial Bash strict mode - https://news.ycombinator.com/item?id=8054440 https://news.ycombinator.com/item?id=8054440 - July 2014 (36 comments)
jenders 1y ago
As a seasoned shell programmer, when I see set -euo pipefail, I know immediately that low quality code “batched list of commands”-type code follows. It’s bad code smell and an anti-pattern as far as I’m concerned. Anyone who has actually has taken the time to learn POSIX/BASH shell to any degree or complexity knows that it’s fully capable of catching errors in-band and that a .sh file isn’t just a dumping ground for repeating your shell history.
- matheusmoreira 1y ago
  > a .sh file isn’t just a dumping ground for repeating your shell history Maybe it should be? As fun as it can be to write complex software in these legacy Turing tarpits, results aren't that great no matter how hard we try. Makefiles are another example.
- naruhodo 1y ago
  For anyone looking for supporting evidence: https://www.reddit.com/r/commandline/comments/g1vsxk/the_first_two_statements_of_your_bash_script/fniifmk/ https://www.reddit.com/r/commandline/comments/g1vsxk/the_fir... "set -euo pipefail" comes with footguns.
gorgoiler 1y ago
The IFS part is misguided. If the author used double quotes around the array reference then words are kept intact: vs=("a b" "c d") for v in "${vs[@]}" do #^♥ ♥^ echo "$v" done #= a b #= c d Whereas in their (counter-)example, with missing quotes: vs=("a b" "c d") for v in ${vs[@]} do #^! !^ echo "$v" done #= a #= b #= c #= d To paraphrase the manual: Any element of an array may be referenced using ${name[subscript]}. If subscript is @ the word expands to all members of name. If the word is double-quoted, "${name[@]}" expands each element of name to a separate word.
- degamad 1y ago
  The author addresses that in footnote 2: > [2] Another approach: instead of altering IFS, begin the loop with for arg in "$@" - double quoting the iteration variable. This changes loop semantics to produces the nicer behavior, and even handles a few edge cases better. The big problem is maintainability. It's easy for even experienced developers to forget to put in the double quotes. Even if the original author has managed to impeccably ingrain the habit, it's foolish to expect that all future maintainers will. In short, relying on quotes has a high risk of introducing subtle time-bomb bugs. Setting IFS renders this impossible.
  - gorgoiler 1y ago
    Thanks, I didn’t see that. I still think the advice is misguided. Double-quote semantics are a fundamental and important part of getting shell scripting right. Trying to bend the default settings so that they are more forgiving of mistakes feels worse than simply fixing those mistakes. In terms of maintainability, fiddling with IFS feels awkward. It’s definitely something you’ll need to teach to anyone unfamiliar with your code. Teach them how "" and @ work, instead! (I agree about maintenance being hard. sh and execve() are a core part of the UNIX API but, as another comment here suggests, for anything complex and long-lived it’s important to get up to a higher level language as soon as you can.)
lblume 1y ago
> For now, I'm still seeking a more satisfactory solution. Please contact me if you have a suggestion. Why not simply end each script with true, just as it is started with a shebang and strict mode? Doesn't sound like the worst solution to me.
Intralexical 1y ago
Beware that `set -e`/errexit has quite a few pitfalls. For example, it's automatically disabled in compound commands that are followed by AND-OR command chains. This can especially bite you if you define a function which you use as a single command. The function's behavior changes based on what you call after it, masking errors that occurred inside it. $ myfunc()(set -e; echo "BEFORE ERROR"; false; echo "AFTER ERROR") $ myfunc BEFORE ERROR $ myfunc || echo "REPORT ERROR!" BEFORE ERROR AFTER ERROR It also doesn't always propagate to subshells. (There might be an additional, non-portable option to make it propagate, though.) $ (set -euo pipefail; x="$(echo A; false; echo B)"; echo "$x") A B EXCEPT for when it sorta does respect the subshell status anyway: $ (set -euo pipefail; x="$(echo A; false; echo B; false)"; echo "$x") <no output> (I think what's happening here is that the assignment statement gets the exit value of the subshell, which is the last command executed in it, without errexit. Subshell completes but top level shell exits.) And Gods help you if you want to combine that with the `local` keyword, which also masks subshell return values: $ f()(set -euo pipefail; local x="$(echo A; false; echo B; false)"; echo "$x"); f A B $ f()(set -euo pipefail; local x; x="$(echo A; false; echo B; false)"; echo "$x"); f <no output> I like shell scripts. I've yet to find anything with more convenient ergonomics for quickly bashing together different tools. But they're full of footguns if you need them to be truly robust.

48 points by dcminter 1y ago

Or just don't use Bash. Python is a great scripting language, and won't blow your foot off if you try to iterate through an array. Other than that, yeah, if you must use bash, set -eu -o pipefail; the IFS is new and mildly interesting idea to me. > The idea is that if a reference is made at runtime to an undefined variable, bash has a syntax for declaring a default value, using the ":-" operator: Just note that defaulting an undefined variable to a value (let's use a default value of "fallback") for these examples is, ${foo-fallback} The syntax, ${foo:-fallback} means "use 'fallback' if foo is unset or is equal to "". (The :, specifically triggers this; there's a bunch of others, like +, which is "use alternate value", or, you'll get the value if the parameter is defined, nothing otherwise. if [[ "${foo+set}" == "set" ]]; then # foo is not undefined. fi And similarly, ${foo:+triggered} will emit triggered if foo is set and not empty.) See "Parameter Expansion" in the manual. I hate this syntax, but it is the syntax one must use to check for undefined-ness.

xelxebar 1y ago
Permit me to vent just a bit: > Python is a great scripting language, and won't blow your foot off if you try to iterate through an array. I kind of hate that every time the topic of shell scripting comes up, we get a troop of comments touting this mindless nonsense. Python has footguns, too. Heck, it's absolutely terrible and hacky if you try to do concatenative programming with it. Does that mean it should never be used? Instead of bashing the language, why not learn bash the language? IME, most of the industry has just absorbed shell programming haphazardly through osmosis, and almost always tries to shove the square pegs of OOP and FP into the round hole that is bash. No wonder people are having a bad time. In contrast, a data-first design that heavily normalizes data into line-oriented tables and passes information around in pipes results in simple, direct code IME. Stop trying to use arrays and embrace data normalization and text. Also, a lot of pain comes from simply not learning the facilities, e.g. the set builtin obviates most uses of string munging and exec: set -- "$@" --file 'filename with spaces.pdf' set -- "$@" 'data|blob with "dangerous" characters' set -- "$@" "$etc" some_command "$@" Anyway, the senseless bash hate is somewhat of a pet peeve of mine. Exunt.
- 3eb7988a1663 1y ago
  All languages have foot guns, but bash is on the more explodey end of the scale. It is not senseless to note that if you can use a safer tool, you should consider it. C/C++ got us really far, but greenfield projects are moving to safer languages where they can. Expert low level programmers, armed with all of the available linting tools are still making unfortunate mistakes. At some point we should switch to something better.
  - oguz-ismail 1y ago
    > At some point we should switch to something better. Agree. Python isn't it though
  - xelxebar 1y ago
    In my years of reading and writing bash as well as Python for sysops tasks, I'd say that bash is the more reliable workhorse of the two. Python tends to encourage a kind of overengineering, resulting in more bugs overall. Many times I've seen hundreds of lines of Python or Typescript result from the attempt to replace just a few lines of bash! The senselessness I object to is not the conscientious choice of tooling or discussion of the failings thereof; it's the fact that every single bash article on here sees the same religious refrain, "Python is better than bash. Period." It's like if every article about vim saw a flood of comments claiming that vim is okay for light editing, but for any real programming we should use a real editor like emacs. If you open vim expecting emacs but with a few different bindings, then it might just explode in you face. If you use bash expecting to be able to program just like Python but with slightly different syntax, then it's not surprising to feel friction. IME, bash works exceptionally well using a data-oriented, text-first design to program architecture. It's just unfortunate that very little of the industry is even aware of this style of programming.

matheusmoreira 1y ago

This honestly should be the default for all scripts. There are so many little annoyances in bash that would make it great if they were changed and improved. Sadly there's just no changing certain things. My number one wishlist feature was a simple library system. Essentially just let me source files by name by searching in some standard user location. I actually wrote and submitted patches for this but it just didn't work out. Maintained my own version for a while and it was nice but not enough to justify the maintenance burden. Bash's number one feature is being the historical default shell on virtually every Linux distribution, without that there's no point. At least we've got shellcheck.

dhamidi 1y ago
The `source` builtin searches through all directories on PATH, not quite a library system, but useable with a bit of discipline in the to-be-sourced shellscripts.
- matheusmoreira 1y ago
  Yeah. I sent patches that added a flag that made it search special library paths instead. source -l some-library
abathur 1y ago
When you say library system, do you mean something more or less like a separate search path and tools for managing it? I've written a little about how we can more or less accomplish something like meaningfully-reusable shell libraries in the nix ecosystem. I think https://www.t-ravis.com/post/shell/the_missing_comprehensive_package_manager_for_shell/ https://www.t-ravis.com/post/shell/the_missing_comprehensive... lays out the general idea but you can also pick through how my bashrc integrates libraries/modules (https://github.com/abathur/bashrc.nix https://github.com/abathur/bashrc.nix). (I'm just dropping these in bin, since that works with existing search path for source. Not ideal, but the generally explicit nature of dependencies in nix minimizes how much things can leak where they aren't meant to go.)
- 1y ago

heybrendan 1y ago

From 2014 [1]. This seems to be posted once per year (at least); however, hardly a complaint as the discussion tends to be high quality. My personal mantra is if it's over 10~20 lines, I should arguably be using another language, like Python (and perhaps correspondingly, subprocess [2], if I'm in a hurry). [1] https://web.archive.org/web/20140523002853/http://redsymbol.net/articles/unofficial-bash-strict-mode/ https://web.archive.org/web/20140523002853/http://redsymbol.... [2] https://docs.python.org/3/library/subprocess.html https://docs.python.org/3/library/subprocess.html

esafak 1y ago
It's time to stop using these archaic shells, and make sure the newer ones are packaged for and included with popular operating systems. Better scripting languages have been offered in shells for a long time: https://en.wikipedia.org/wiki/Scsh https://en.wikipedia.org/wiki/Scsh These days I've settled on https://www.nushell.sh/ https://www.nushell.sh/
- tux1968 1y ago
  I think https://oils.pub/ https://oils.pub/ has a good shot at being the eventual replacement because it has a very strong transition story. Being backward compatible, while allowing you to progressively embrace a modern replacement, is pretty powerful.
- ZYbCRq22HbJ2y7 1y ago
  i don't think nushell is better than bash for day to day things. it is nice when you need it though, and then you can just run it, like with any shell.
  - esafak 1y ago
    Why would you do that when nushell is strictly better? It's bash that I drop into when I rarely need to, which is when someone gives a bash recipe, not nushell.

chubot 1y ago

I use essentially this, but I think this post is over 10 years old (needs a date), and it's now INCOMPLETE. bash introduced an option to respect rather than ignore errors within command sub processes years ago. So if you want to be safer, do something like: #!/bin/bash set -euo pipefail shopt -s inherit_errexit That works as-is in OSH, which is part of https://oils.pub/ https://oils.pub/ (edit: the last time this came up was a year ago, and here's a more concrete example - https://lobste.rs/s/1wohaz/posix_2024_changes#c_9oo1av https://lobste.rs/s/1wohaz/posix_2024_changes#c_9oo1av ) --- But that's STILL incomplete because POSIX mandates that errors be LOST. That is, it mandates broken error handling. For example, there what I call the "if myfunc" pitfall set -e my-deploy-func # errors respected if ! my-deploy-func; then # errors lost echo failed fi my-deploy-func || echo fail # errors lost But even if you fix that, it's still not enough. --- I describe all the problems in this doc, e.g. waiting for process subs: YSH Fixes Shell's Error Handling (errexit) - https://oils.pub/release/latest/doc/error-handling.html https://oils.pub/release/latest/doc/error-handling.html Summary: YSH fixes all shell error handling issues. This was surprisingly hard and required many iterations, but it has stood up to scrutiny. For contrast, here is a recent attempt at fixing bash, which is also incomplete, and I argue is a horrible language design: https://lobste.rs/s/kidktn/bash_patch_add_shopt_for_implicit_place https://lobste.rs/s/kidktn/bash_patch_add_shopt_for_implicit...

aidenn0 1y ago
What are your thoughts on pipefail in bash? I know ysh (and maybe osh too?) are smart about a -13 result from a pipe, but I stopped using pipefail since the return value can be based on a race, causing things to randomly not work at some point in the future. [edit] Per the link you posted osh treats a -13 result as success with sigpipe_status_ok, but I'm still interested in your thoughts on if pipefail is better or worse to recommend as "always on" for bash.
- chubot 1y ago
  All my scripts use pipefail, and I haven't run into problems I think `head` tends to cause SIGPIPE in practice, and that is what sigpipe_status_ok is for but I stopped using pipefail since the return value can be based on a race, causing things to randomly not work at some point in the future. Hm I'd be interested in an example of this The order that processes in a pipeline finish is non-deterministic, but that should NOT affect the status, with either pipefail or not. That is, the status should be deterministic ... it's possible that SIGPIPE is an exception to that, though that wouldn't affect YSH.

kayson 1y ago

Or use shellcheck: https://www.shellcheck.net/ https://www.shellcheck.net/

burnt-resistor 1y ago
Tl;dr: Use both because they aren't mutex. Shellcheck isn't a complete solution, and running -e mode is essential to smaller bash files. Shellcheck even knows if a script is in -e mode or not.

cendyne 1y ago

If you ever try something like this with another bash-hackery tool like NVM, you will have a very bad time.

jacob2161 1y ago

Stylistically, I much prefer #!/bin/bash set -o errexit set -o nounset set -o pipefail It reminds me when I wrote a lot of Perl: #!/usr/bin/perl use strict; use warnings; I also prefer --long --args everywhere possible and a comment on any short flag unless it's incredibly common. I've been writing all my Bash scripts like this for ~15 years and it's definitely the way to go.

Waterluvian 1y ago
Yeah! I feel like long args very strongly fits with the overall idea of this blog post: when you are running a script you care about different things and want different behaviours than when you’re working interactively. The short args are great for when you’re typing into a terminal. But when you’re writing and saving a script, be descriptive! One of the most irritating things about bash (and Linux, I guess?) is how magical all the -FEU -lh 3 -B 1 incantations are. They give off a vibe I call “nerd snobbish” where it’s a sort of “oh you don’t know what that all means? How fretful!”
burnt-resistor 1y ago
Too damn verbose and you're assuming bash is at /bin. This will cause problems in nix and other environments where bash should be found on the path instead. Always, always unless the script is guaranteed to not be portable to other platforms: #!/usr/bin/env bash ...
- jacob2161 1y ago
  You're assuming env is in /usr/bin? There are real environments where it's at /bin/env. Running a script with bash script.sh Works on any system with bash in the path, doesn't require that it be executable, and is usually the ideal way to execute them. I don't really believe in trying to write portable Bash scripts. There's a lot more to it than just getting the shebang right. Systems and commands tend to have subtle differences and flags are often incompatible. Lots of branching logic makes them hideous quickly. I prefer to write a script for each platform or just write a small portable Go program and compile it for each platform. I'm almost always writing Bash scripts for Linux that run in Debian-based containers, where using /bin/bash is entirely correct and safe. Regarding verbosity, most people come to appreciate a little verbosity in exchange for clarity with more experience. Clever and compact code is almost never what you want in production code.

vivzkestrel 1y ago

Again another great read about why you should NOT use this https://mywiki.wooledge.org/BashFAQ/105 https://mywiki.wooledge.org/BashFAQ/105

vivzkestrel 1y ago

Lots of arguments against not using set -euo pipefail https://www.reddit.com/r/commandline/comments/g1vsxk/comment/fniifmk/ https://www.reddit.com/r/commandline/comments/g1vsxk/comment... anything you wanna say about this?

awestroke 1y ago
euo pipefail has been the one good thing with bash. I'll start looking at alternatives now
jacob2161 1y ago
That post is quite nitpicky, pointing to edge cases and old version behavior. The essence of it is that writing reliable Bash scripts requires significant domain knowledge, which is true. Bash scripts are great if you've been doing it for a very long time or you're doing something simple and small (<50 LOC). If it's complicated or large, you should just write it in a proper programming language anyway.
- heresie-dabord 1y ago
  > something simple and small (<50 LOC). If it's complicated or large, you should just write it in a proper programming language anyway. Regardless of LOC, eject from bash/awk as soon as you need a data structure, and choose a better language.
  - jacob2161 1y ago
    Yeah, that's a good point. It's less about LOC than complexity (data structures, complicated string parsing, handling binary data, etc). And that's actually the advice I follow myself. Some longish Bash scripts are very maintainable and reliable, because they're still quite regular and simple.

burnt-resistor 1y ago

set -Eeuo pipefail It's missing -E to respect traps.

tomhow 1y ago

Use the unofficial Bash strict mode - https://news.ycombinator.com/item?id=8054440 https://news.ycombinator.com/item?id=8054440 - July 2014 (36 comments)

jenders 1y ago

As a seasoned shell programmer, when I see set -euo pipefail, I know immediately that low quality code “batched list of commands”-type code follows. It’s bad code smell and an anti-pattern as far as I’m concerned. Anyone who has actually has taken the time to learn POSIX/BASH shell to any degree or complexity knows that it’s fully capable of catching errors in-band and that a .sh file isn’t just a dumping ground for repeating your shell history.

matheusmoreira 1y ago
> a .sh file isn’t just a dumping ground for repeating your shell history Maybe it should be? As fun as it can be to write complex software in these legacy Turing tarpits, results aren't that great no matter how hard we try. Makefiles are another example.
naruhodo 1y ago
For anyone looking for supporting evidence: https://www.reddit.com/r/commandline/comments/g1vsxk/the_first_two_statements_of_your_bash_script/fniifmk/ https://www.reddit.com/r/commandline/comments/g1vsxk/the_fir... "set -euo pipefail" comes with footguns.

gorgoiler 1y ago

The IFS part is misguided. If the author used double quotes around the array reference then words are kept intact: vs=("a b" "c d") for v in "${vs[@]}" do #^♥ ♥^ echo "$v" done #= a b #= c d Whereas in their (counter-)example, with missing quotes: vs=("a b" "c d") for v in ${vs[@]} do #^! !^ echo "$v" done #= a #= b #= c #= d To paraphrase the manual: Any element of an array may be referenced using ${name[subscript]}. If subscript is @ the word expands to all members of name. If the word is double-quoted, "${name[@]}" expands each element of name to a separate word.

degamad 1y ago
The author addresses that in footnote 2: > [2] Another approach: instead of altering IFS, begin the loop with for arg in "$@" - double quoting the iteration variable. This changes loop semantics to produces the nicer behavior, and even handles a few edge cases better. The big problem is maintainability. It's easy for even experienced developers to forget to put in the double quotes. Even if the original author has managed to impeccably ingrain the habit, it's foolish to expect that all future maintainers will. In short, relying on quotes has a high risk of introducing subtle time-bomb bugs. Setting IFS renders this impossible.
- gorgoiler 1y ago
  Thanks, I didn’t see that. I still think the advice is misguided. Double-quote semantics are a fundamental and important part of getting shell scripting right. Trying to bend the default settings so that they are more forgiving of mistakes feels worse than simply fixing those mistakes. In terms of maintainability, fiddling with IFS feels awkward. It’s definitely something you’ll need to teach to anyone unfamiliar with your code. Teach them how "" and @ work, instead! (I agree about maintenance being hard. sh and execve() are a core part of the UNIX API but, as another comment here suggests, for anything complex and long-lived it’s important to get up to a higher level language as soon as you can.)

lblume 1y ago

> For now, I'm still seeking a more satisfactory solution. Please contact me if you have a suggestion. Why not simply end each script with true, just as it is started with a shebang and strict mode? Doesn't sound like the worst solution to me.

Intralexical 1y ago

Beware that `set -e`/errexit has quite a few pitfalls. For example, it's automatically disabled in compound commands that are followed by AND-OR command chains. This can especially bite you if you define a function which you use as a single command. The function's behavior changes based on what you call after it, masking errors that occurred inside it. $ myfunc()(set -e; echo "BEFORE ERROR"; false; echo "AFTER ERROR") $ myfunc BEFORE ERROR $ myfunc || echo "REPORT ERROR!" BEFORE ERROR AFTER ERROR It also doesn't always propagate to subshells. (There might be an additional, non-portable option to make it propagate, though.) $ (set -euo pipefail; x="$(echo A; false; echo B)"; echo "$x") A B EXCEPT for when it sorta does respect the subshell status anyway: $ (set -euo pipefail; x="$(echo A; false; echo B; false)"; echo "$x") <no output> (I think what's happening here is that the assignment statement gets the exit value of the subshell, which is the last command executed in it, without errexit. Subshell completes but top level shell exits.) And Gods help you if you want to combine that with the `local` keyword, which also masks subshell return values: $ f()(set -euo pipefail; local x="$(echo A; false; echo B; false)"; echo "$x"); f A B $ f()(set -euo pipefail; local x; x="$(echo A; false; echo B; false)"; echo "$x"); f <no output> I like shell scripts. I've yet to find anything with more convenient ergonomics for quickly bashing together different tools. But they're full of footguns if you need them to be truly robust.