7 ms·

I consider shellcheck absolutely essential if you're writing even a single line of Bash. I also start all my scripts with this "unofficial bash strict mode" an

I consider shellcheck absolutely essential if you're writing even a single line of Bash. I also start all my scripts with this "unofficial bash strict mode" and DIR= shortcut: #!/usr/bin/env bash ### Bash Environment Setup # http://redsymbol.net/articles/unofficial-bash-strict-mode/ # https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html # set -o xtrace set -o errexit set -o errtrace set -o nounset set -o pipefail IFS=$'\n' DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" I have more tips/tricks here: https://github.com/pirate/bash-utils/blob/master/util/base.sh https://github.com/pirate/bash-utils/blob/master/util/base.s...

rantwasp 6y ago
set -euxo pipefail
- ed25519FUUU 6y ago
  Every script. Good enough.
- Rapzid 6y ago
  Ahh, errexit: unless the command that fails is part of an until or while loop, part of an if statement, part of a && or || list, or if the command's return status is being inverted using !. All hope is lost.
rgrau 6y ago
You've a lot of cool stuff in that repo! I hope you don't mind me 'stealing' some stuff for a kind of booklet I'm working on with several shell tips and tricks. https://github.com/kidd/scripting-field-guide/ https://github.com/kidd/scripting-field-guide/ PS: Any comments/PRs are super welcome there too
- nikisweeting 6y ago
  No problem, it's all MIT licensed so it's not stealing haha. See my parent comment with my full reading list as well. If you want to link back to my repo as a cited source/credit that would be nice, but is not required :)
  - rgrau 6y ago
    Cool thanks! I will link back and credit you :)
ausjke 6y ago
errtrace is new to me, otherwise I do similar things. set -ueEo pipefail
- JdeBP 6y ago
  "pipefail" is also available in the FreeBSD Almquist shell, but not in the Debian Almquist shell.
rustshellscript 6y ago
I have created a similar library for bash. However, there are more pitfalls related to errexit, see [1] and even shellcheck can not help there. I have tried to solve the pitfalls in my library, but it turned out to be ugly and unreliable. That's why I'm trying to use rust [2] for shell scripting like tasks nowadays. [1] https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md#gotcha-errexit-is-ignored-in-command-arguments https://github.com/anordal/shellharden/blob/master/how_to_do... [2] https://github.com/rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib
- maddyboo 6y ago
  Your [1] doesn’t mention it, but I find shopt -s inherit_errexit to be indispensable. https://twitter.com/hellsmaddy/status/1273744824835796993?s=21 https://twitter.com/hellsmaddy/status/1273744824835796993?s=...
abathur 6y ago
I've had a thought percolating (for many months?) but I haven't tried to phrase it and I'm leery it may sound condescending (this is also more of a public address than direct response) ... Shellcheck tends to steer us away from "weird" parts of Bash/shell and will happily suggest changing code that was correct as written. There are good reasons for this (and I still think most scripts/projects should use Shellcheck), but Bash is a weird language, and there's a lot of power/potential in some of the weird stuff Shellcheck walls off. If I had to nail down some heuristics for working with it...: - do lean heavily on Shellcheck for writing new code if you're unfamiliar with the language or are only writing it under duress - don't implement its suggestions in bulk (the time you save on iterating can easily be blown later trying to debug subtle behavior changes) - don't apply them to code you don't understand (if you have the time and interest to understand it but find yourself stuck on some un-searchable syntax, explainshell may help) - don't adopt them without careful testing - do take Shellcheck warnings about code that is correct-as-written as a hint to leave a comment explaining what's going on - do leave Shellcheck off (or only use it intermittently) if you're trying to explore, play, or otherwise learn the language
- sedatk 6y ago
  That advice is applicable to any kind of code analysis tool.
  - abathur 6y ago
    Fair :) but I think there's a degree difference here (and this is more about the language than Shellcheck itself). Also, maybe my original phrasing left room for misinterpretation? Shellcheck will happily suggest changes that break working code. I think I've had it happen ~4 times this year? Obviously it varies a little by tool type and ecosystem--it's obviously not much of a surprise if a dead-code analysis tool suggests removing things you can't actually cut, and likewise linters in most languages can point out unused variables/parameters that you can't cut. But I can't recall the last time another tool gave me specific do-x-not-y suggestions that just weren't anywhere close to fungible.
    - 6y ago
draebek 6y ago
Note that this same wiki has a good page on the numerous surprising behaviors of errexit: https://mywiki.wooledge.org/BashFAQ/105 https://mywiki.wooledge.org/BashFAQ/105 My perception is that there is no consensus on whether or not to use it. I go back and forth, but I probably use it more often than not.
- nikisweeting 6y ago
  imho all the behaviors that post outlines as "confusing" are my desired behaviors most of the time. For me it's pretty clear-cut, error out by default and only continue when explicitly allowed with something like `|| true`.
BiteCode_dev 6y ago
Same, but I also do "shopt -s dotglob" so that * includes hidden files.
- hoseja 6y ago
  That's really fun to pronounce.
- nikisweeting 6y ago
  ooh good one, I'll add that to my snippet
nikisweeting 6y ago
Here's my full bash reading list: - ShellCheck: life-changing BASH linter and testing toolkit https://github.com/koalaman/shellcheck https://github.com/koalaman/shellcheck - 30 interesting commands for the Linux shell – Víctor López Ferrando https://www.lopezferrando.com/30-interesting-shell-commands/ https://www.lopezferrando.com/30-interesting-shell-commands/ - 7 Surprising Bash Variables https://zwischenzugs.com/2019/05/11/seven-surprising-bash-variables/ https://zwischenzugs.com/2019/05/11/seven-surprising-bash-va... - anordal/shellharden https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md https://github.com/anordal/shellharden/blob/master/how_to_do... - barryclark/bashstrap https://github.com/barryclark/bashstrap https://github.com/barryclark/bashstrap - BashPitfalls : Greg's Wiki http://mywiki.wooledge.org/BashPitfalls http://mywiki.wooledge.org/BashPitfalls - Common shell script mistakes http://www.pixelbeat.org/programming/shell_script_mistakes.html http://www.pixelbeat.org/programming/shell_script_mistakes.h... - Comparison of all the UNIX shells http://hyperpolyglot.org/unix-shells http://hyperpolyglot.org/unix-shells - Defensive Bash Programming https://www.kfirlavi.com/blog/2012/11/14/defensive-bash-programming/index.html https://www.kfirlavi.com/blog/2012/11/14/defensive-bash-prog... or https://jonlabelle.com/snippets/view/markdown/defensive-bash-programming https://jonlabelle.com/snippets/view/markdown/defensive-bash... - Bash FAQ and Cookbook https://mywiki.wooledge.org/BashFAQ https://mywiki.wooledge.org/BashFAQ - Detecting the use of "curl | bash" server side https://idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side https://idontplaydarts.com/2016/04/detecting-curl-pipe-bash-... - Gensokyo Blog - Use Bash Builtins shell,fish,bash https://blog.gensokyo.io/a/fafbe742.html https://blog.gensokyo.io/a/fafbe742.html - Rich’s sh (POSIX shell) tricks http://www.etalabs.net/sh_tricks.html http://www.etalabs.net/sh_tricks.html - Shell Scripts Matter https://dev.to/thiht/shell-scripts-matter https://dev.to/thiht/shell-scripts-matter - Shell Style Guide https://google.github.io/styleguide/shell.xml https://google.github.io/styleguide/shell.xml - Shellcode Injection - Dhaval Kapil https://dhavalkapil.com/blogs/Shellcode-Injection/ https://dhavalkapil.com/blogs/Shellcode-Injection/ - Something you didn't know about functions in bash http://catonmat.net/blog/bash-functions http://catonmat.net/blog/bash-functions - Ten More Things I Wish I’d Known About bash https://zwischenzugs.com/2018/01/21/ten-more-things-i-wish-id-known-about-bash https://zwischenzugs.com/2018/01/21/ten-more-things-i-wish-i... - Ten Things I Wish I’d Known About bash https://zwischenzugs.com/2018/01/06/ten-things-i-wish-id-known-about-bash https://zwischenzugs.com/2018/01/06/ten-things-i-wish-id-kno... - Testing Bash scripts with BATS https://opensource.com/article/19/2/testing-bash-bats https://opensource.com/article/19/2/testing-bash-bats - Testing Bash scripts with Critic.sh https://github.com/Checksum/critic.sh https://github.com/Checksum/critic.sh - Useful BASH and UNIX commands https://cb.vu/unixtoolbox.xhtml https://cb.vu/unixtoolbox.xhtml - When Bash Scripts Bite :: Jane Street Tech Blogs https://blogs.janestreet.com/when-bash-scripts-bite/ https://blogs.janestreet.com/when-bash-scripts-bite/ - Bashible: Ansible-like framework for bash-based devops https://github.com/mig1984/bashible https://github.com/mig1984/bashible - Auto-parse help text from comment at the top of script https://samizdat.dev/help-message-for-shell-scripts/ https://samizdat.dev/help-message-for-shell-scripts/ - Make bash scripts safer by writing them in Rust https://github.com/rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib - Additional shell options for non-trivial bash scripts https://saveriomiroddi.github.io/Additional-shell-options-for-non-trivial-bash-shell-scripts/ https://saveriomiroddi.github.io/Additional-shell-options-fo...

rantwasp 6y ago

set -euxo pipefail

ed25519FUUU 6y ago
Every script. Good enough.
Rapzid 6y ago
Ahh, errexit: unless the command that fails is part of an until or while loop, part of an if statement, part of a && or || list, or if the command's return status is being inverted using !. All hope is lost.

rgrau 6y ago

You've a lot of cool stuff in that repo! I hope you don't mind me 'stealing' some stuff for a kind of booklet I'm working on with several shell tips and tricks. https://github.com/kidd/scripting-field-guide/ https://github.com/kidd/scripting-field-guide/ PS: Any comments/PRs are super welcome there too

nikisweeting 6y ago
No problem, it's all MIT licensed so it's not stealing haha. See my parent comment with my full reading list as well. If you want to link back to my repo as a cited source/credit that would be nice, but is not required :)
- rgrau 6y ago
  Cool thanks! I will link back and credit you :)

ausjke 6y ago

errtrace is new to me, otherwise I do similar things. set -ueEo pipefail

JdeBP 6y ago
"pipefail" is also available in the FreeBSD Almquist shell, but not in the Debian Almquist shell.

rustshellscript 6y ago

I have created a similar library for bash. However, there are more pitfalls related to errexit, see [1] and even shellcheck can not help there. I have tried to solve the pitfalls in my library, but it turned out to be ugly and unreliable. That's why I'm trying to use rust [2] for shell scripting like tasks nowadays. [1] https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md#gotcha-errexit-is-ignored-in-command-arguments https://github.com/anordal/shellharden/blob/master/how_to_do... [2] https://github.com/rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib

maddyboo 6y ago
Your [1] doesn’t mention it, but I find shopt -s inherit_errexit to be indispensable. https://twitter.com/hellsmaddy/status/1273744824835796993?s=21 https://twitter.com/hellsmaddy/status/1273744824835796993?s=...

abathur 6y ago

I've had a thought percolating (for many months?) but I haven't tried to phrase it and I'm leery it may sound condescending (this is also more of a public address than direct response) ... Shellcheck tends to steer us away from "weird" parts of Bash/shell and will happily suggest changing code that was correct as written. There are good reasons for this (and I still think most scripts/projects should use Shellcheck), but Bash is a weird language, and there's a lot of power/potential in some of the weird stuff Shellcheck walls off. If I had to nail down some heuristics for working with it...: - do lean heavily on Shellcheck for writing new code if you're unfamiliar with the language or are only writing it under duress - don't implement its suggestions in bulk (the time you save on iterating can easily be blown later trying to debug subtle behavior changes) - don't apply them to code you don't understand (if you have the time and interest to understand it but find yourself stuck on some un-searchable syntax, explainshell may help) - don't adopt them without careful testing - do take Shellcheck warnings about code that is correct-as-written as a hint to leave a comment explaining what's going on - do leave Shellcheck off (or only use it intermittently) if you're trying to explore, play, or otherwise learn the language

sedatk 6y ago
That advice is applicable to any kind of code analysis tool.
- abathur 6y ago
  Fair :) but I think there's a degree difference here (and this is more about the language than Shellcheck itself). Also, maybe my original phrasing left room for misinterpretation? Shellcheck will happily suggest changes that break working code. I think I've had it happen ~4 times this year? Obviously it varies a little by tool type and ecosystem--it's obviously not much of a surprise if a dead-code analysis tool suggests removing things you can't actually cut, and likewise linters in most languages can point out unused variables/parameters that you can't cut. But I can't recall the last time another tool gave me specific do-x-not-y suggestions that just weren't anywhere close to fungible.
  - 6y ago

draebek 6y ago

Note that this same wiki has a good page on the numerous surprising behaviors of errexit: https://mywiki.wooledge.org/BashFAQ/105 https://mywiki.wooledge.org/BashFAQ/105 My perception is that there is no consensus on whether or not to use it. I go back and forth, but I probably use it more often than not.

nikisweeting 6y ago
imho all the behaviors that post outlines as "confusing" are my desired behaviors most of the time. For me it's pretty clear-cut, error out by default and only continue when explicitly allowed with something like `|| true`.

BiteCode_dev 6y ago

Same, but I also do "shopt -s dotglob" so that * includes hidden files.

hoseja 6y ago
That's really fun to pronounce.
nikisweeting 6y ago
ooh good one, I'll add that to my snippet

nikisweeting 6y ago

Here's my full bash reading list: - ShellCheck: life-changing BASH linter and testing toolkit https://github.com/koalaman/shellcheck https://github.com/koalaman/shellcheck - 30 interesting commands for the Linux shell – Víctor López Ferrando https://www.lopezferrando.com/30-interesting-shell-commands/ https://www.lopezferrando.com/30-interesting-shell-commands/ - 7 Surprising Bash Variables https://zwischenzugs.com/2019/05/11/seven-surprising-bash-variables/ https://zwischenzugs.com/2019/05/11/seven-surprising-bash-va... - anordal/shellharden https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md https://github.com/anordal/shellharden/blob/master/how_to_do... - barryclark/bashstrap https://github.com/barryclark/bashstrap https://github.com/barryclark/bashstrap - BashPitfalls : Greg's Wiki http://mywiki.wooledge.org/BashPitfalls http://mywiki.wooledge.org/BashPitfalls - Common shell script mistakes http://www.pixelbeat.org/programming/shell_script_mistakes.html http://www.pixelbeat.org/programming/shell_script_mistakes.h... - Comparison of all the UNIX shells http://hyperpolyglot.org/unix-shells http://hyperpolyglot.org/unix-shells - Defensive Bash Programming https://www.kfirlavi.com/blog/2012/11/14/defensive-bash-programming/index.html https://www.kfirlavi.com/blog/2012/11/14/defensive-bash-prog... or https://jonlabelle.com/snippets/view/markdown/defensive-bash-programming https://jonlabelle.com/snippets/view/markdown/defensive-bash... - Bash FAQ and Cookbook https://mywiki.wooledge.org/BashFAQ https://mywiki.wooledge.org/BashFAQ - Detecting the use of "curl | bash" server side https://idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side https://idontplaydarts.com/2016/04/detecting-curl-pipe-bash-... - Gensokyo Blog - Use Bash Builtins shell,fish,bash https://blog.gensokyo.io/a/fafbe742.html https://blog.gensokyo.io/a/fafbe742.html - Rich’s sh (POSIX shell) tricks http://www.etalabs.net/sh_tricks.html http://www.etalabs.net/sh_tricks.html - Shell Scripts Matter https://dev.to/thiht/shell-scripts-matter https://dev.to/thiht/shell-scripts-matter - Shell Style Guide https://google.github.io/styleguide/shell.xml https://google.github.io/styleguide/shell.xml - Shellcode Injection - Dhaval Kapil https://dhavalkapil.com/blogs/Shellcode-Injection/ https://dhavalkapil.com/blogs/Shellcode-Injection/ - Something you didn't know about functions in bash http://catonmat.net/blog/bash-functions http://catonmat.net/blog/bash-functions - Ten More Things I Wish I’d Known About bash https://zwischenzugs.com/2018/01/21/ten-more-things-i-wish-id-known-about-bash https://zwischenzugs.com/2018/01/21/ten-more-things-i-wish-i... - Ten Things I Wish I’d Known About bash https://zwischenzugs.com/2018/01/06/ten-things-i-wish-id-known-about-bash https://zwischenzugs.com/2018/01/06/ten-things-i-wish-id-kno... - Testing Bash scripts with BATS https://opensource.com/article/19/2/testing-bash-bats https://opensource.com/article/19/2/testing-bash-bats - Testing Bash scripts with Critic.sh https://github.com/Checksum/critic.sh https://github.com/Checksum/critic.sh - Useful BASH and UNIX commands https://cb.vu/unixtoolbox.xhtml https://cb.vu/unixtoolbox.xhtml - When Bash Scripts Bite :: Jane Street Tech Blogs https://blogs.janestreet.com/when-bash-scripts-bite/ https://blogs.janestreet.com/when-bash-scripts-bite/ - Bashible: Ansible-like framework for bash-based devops https://github.com/mig1984/bashible https://github.com/mig1984/bashible - Auto-parse help text from comment at the top of script https://samizdat.dev/help-message-for-shell-scripts/ https://samizdat.dev/help-message-for-shell-scripts/ - Make bash scripts safer by writing them in Rust https://github.com/rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib - Additional shell options for non-trivial bash scripts https://saveriomiroddi.github.io/Additional-shell-options-for-non-trivial-bash-shell-scripts/ https://saveriomiroddi.github.io/Additional-shell-options-fo...