9 ms·
Common shell script mistakes (2008)
- illicium 10y agoShout out to Shellcheck (http://www.shellcheck.net/ http://www.shellcheck.net/), a linter for shell scripts
- _lce0 10y agoIf you're on Sublime, this works like a charm! https://github.com/SublimeLinter/SublimeLinter-shellcheck https://github.com/SublimeLinter/SublimeLinter-shellcheck
- jasonm23 10y agoIf you're on Vim or Emacs you already (maybe) know you have integration with shellcheck Emacs - https://github.com/koalaman/shellcheck https://github.com/koalaman/shellcheck Vim - part of syntactic - https://github.com/scrooloose/syntastic/blob/master/syntax_checkers/sh/shellcheck.vim https://github.com/scrooloose/syntastic/blob/master/syntax_c...
- ourmandave 10y agoWritten in 2008.
- dang 10y agoThanks; added.
- gberger 10y agoShell hasn't changed much, if at all, since, has it?
- pwd_mkdb 10y agonope. reminds me of comments where someone is purporting to be able to assess the quality of software based only on looking to see when the last changes to the source code were made.
- eximius 10y agoI'm not saying that is a valid strategy, but man I see more bad old code than new code.
- deleted 10y ago[deleted]
- pvg 10y agoThe google shell style guide linked now lives here and is probably an even better starting point than this article: https://google.github.io/styleguide/shell.xml https://google.github.io/styleguide/shell.xml
- pmarin 10y agoThe google shell style uses bash which is not portable, one of the main point of the OP article.
- nailer 10y agoBash is the default shell on the most popular server nix (Linux) and the most desktop nix (OS X, though it's always a slightly old version). If you use OpenBSD and hate ports, then maybe ignore all the good things bash brings and write everything in Bourne shell. If not, then write bash. Just fire it up with '/usr/bin/env bash' to label it correctly.
- deleted 10y ago[deleted]
- Someone 10y ago"OS X, though it's always a slightly old version" Apple ships the last version that had a GPL2 license. That version gets older by a day every day. On OS X 10.11.2, it seems to be bash 3.2 (http://opensource.apple.com/source/bash/bash-99/ http://opensource.apple.com/source/bash/bash-99/, via http://opensource.apple.com/release/os-x-10112/ http://opensource.apple.com/release/os-x-10112/), which is from October 12, 2006 (https://lists.gnu.org/archive/html/info-gnu/2006-10/msg00006.html https://lists.gnu.org/archive/html/info-gnu/2006-10/msg00006...) I wouldn't call that slightly old.
- JdeBP 10y ago> Bash is the default shell Not for nigh on a decade, now. * https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh * https://wiki.ubuntu.com/DashAsBinSh/Spec https://wiki.ubuntu.com/DashAsBinSh/Spec * https://lists.debian.org/debian-release/2007/07/msg00027.html https://lists.debian.org/debian-release/2007/07/msg00027.htm... * https://wiki.debian.org/DashAsBinSh https://wiki.debian.org/DashAsBinSh
- olalonde 10y agoIs there a modern "JavaScript: The Good Parts" for bash? There are so many ways to do things and it's often hard to tell which is preferable.
- arthulia 10y agoNot sure if this is what you're looking for, but it's where I tend to go when writing something. http://mywiki.wooledge.org/BashGuide http://mywiki.wooledge.org/BashGuide
- stirner 10y agoI'm a big fan of this guide. I wish every language had such a clear best practices list. See also http://mywiki.wooledge.org/BashPitfalls http://mywiki.wooledge.org/BashPitfalls
- 0xmohit 10y agoNot exactly "The Good Parts" for bash, but one might want to look at Turtle library: https://github.com/Gabriel439/Haskell-Turtle-Library https://github.com/Gabriel439/Haskell-Turtle-Library Related: http://www.haskellforall.com/2015/01/use-haskell-for-shell-scripting.html http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...
- paulddraper 10y ago> please be wary of writing scripts that deviate from these abstractions, and have significant data manipulation in the shell process itself. I appreciate he mentions this first. Shell scripting is excellent in its domain, just as SQL is great in its. You would write an application in SQL and you shouldn't write one in shell.
- gizi 10y agoBash is the love of my life! I have been working for years on this problem now (not full-time of course), gradually moving in the direction of finally being able to challenge this: "Inappropriate use shell is the main domain specific language designed to manipulate the UNIX abstractions for data and logic, i.e. files and processes. ... Correspondingly, please be wary of writing scripts that deviate from these abstractions, and have significant data manipulation in the shell process itself. While flexible, shell is not designed as a general purpose language and becomes unwieldly when ... " Another person has actually solved the most important show stopper already: http://ctypes.sh http://ctypes.sh. What now remains to be solved, are a few minor, additional details, and then simply writing a good manual of how to very successfully use bash as a general-purpose language. My personal belief is that everything that you can do in other scripting languages, you can also do in Bash, only better.
- nerdponx 10y agoYou're missing out on Zsh ;)
- daveguy 10y ago> My personal belief is that everything that you can do in other scripting languages, you can also do in Bash, only better. Matrix multiplication?
- mbrock 10y agoEffective bash programming involves invoking other Unix commands. That's what the shell is for, after all. I don't know off the top of my head what you'd use for matrix multiplication; I guess it depends. For another example, jq and jshon are excellent tools for querying, transforming, or creating JSON values. My experience is that programming by composing Unix programs that communicate via pipes and files can be very pleasant and productive. Sometimes I lament aspects of bash syntax and semantics, but it's also much more convenient than other scripting languages for tasks related to spawning programs and pipelines and working with files. One of my recent projects that I've been having a lot of fun with is a kind of continuous integration system built mostly with bash scripts. It involves a lot of calls to git and Docker, process manipulation, file descriptor redirection, and file system manipulation—and bash makes all this pretty easy and concise.
- soyiuz 10y agoGreat post. I disagree on using the concise form of the if statement, however. if [ "$var" = "find" ]; then echo "found" fi Is far more readable than its equivalent [ "$var" = "find" ] && echo "found" I understand the upside of readability. What does concision get me?
- pwd_mkdb 10y agoconcision gets me less code to read. that's how i define "readability". but if forum commments are any indication, i know my preferences do not follow the norm. most programmers seems to prefer verbosity. however in my case verbosity slows me down.
- eximius 10y agoSomething very 'readable' is something read and understood easily. Python is considered very readable, for example. I would not consider Python verbose. C++ Template programming is verbose and it is not particularly pleasant to read. At the same, APL or J or something can be very concise and is largely unreadable to most. Notice how Python, which is considered by most to be readable, is somewhere in between.
- vacri 10y agoIf the command is as short as "echo 'found'", then that one-liner is easier, but if it's a nice, lengthy command, I find the 'full' version easier to read.
- teamhappy 10y agoI define readability the same way the Linux kernel style guide does: Avoid tricky expressions.
- wodenokoto 10y agoBy that logic you find minified javascript easier to read too.
- nikita2206 10y ago
- Too 10y ago> Just test the string directly like [ "$var" ] && echo "var not empty" Will this work if you've activated set -u, fail on unset variables? Will the !-z variant do?
- brbsix 10y agoIf `set -u` is enabled, the aforementioned test will fail on unset variables. To avoid this, you can "declare" `var` at the beginning of the script, e.g. `var=''`. Also instead of `[ ! -z "$var" ]` I prefer `[ -n "$var" ]`, which is the same as `[ "$var" ]` but more explicit.
- stewartbutler 10y agoSo many edge cases. I prefer to just use perl if I am doing anything more complicated than making a wrapper for some other executable. Harder to screw up, easier to test, same expressiveness, better portability (works with all shells and with windows), plus you get regular expressions for free. Python is a good alternative as well.
- s_kilk 10y agoPerl is brutally under-rated as a scripting tool these days. I suppose it's passed into software orthodoxy by this point, the notion that perl is a uniformly horrific tool akin to nuclear waste.
- naivepiano 10y agoI think that this is because people are finding Python equally well suited for such tasks and more modern. Perl (easy to) obscure syntax is not helping either.
- jerf 10y agoIf you're doing a really grungy, grotty text mangling task in a UNIX pipeline, suddenly all those crazy Perl features start making sense. The reason why Perl is arguably not the best choice for other tasks, such as creating large systems, is precisely that it's actually got many distinct advantages over Python for that sort of task. It's also really good for "I have a UNIX pipeline but this one chunk of it needs a database and this other chunk needs a JSON file and a few other things that shell just can't do". For instance, for the first time in 15+ years, I recently had a reason to use Perl's half-deprecated, off-handedly-included form printing support, and, you know what? It's really nice if that's what you want. Many dozens of lines of code replaced by what is basically a picture of what I want. Python is a great language, but every once in a while "explicit is better than implicit", which is generally a philosophy I agree with wholeheartedly, results in a lot more "explicit", in the form of "code I had to write", than you might be looking for. (Or, alternatively, you start writing Python that is a lot more implicit, which isn't exactly a huge victory for Python here.) Nowadays I'd also suggest that if you find yourself having a lot of those tasks, you need to ask some questions about how you're storing your data. But pretty much everybody ends up with some of these somewhere; it's hard to eliminate them, and not really worth it (due to diminishing returns).
- bootload 10y ago"Writing shell scripts leaves a lot of room to make mistakes, in ways that will cause your scripts to break on certain input, or (if some input is untrusted) open up security vulnerabilities. Here are some tips on how to make your shell scripts safer. Don't" [0] [0] MIT, writing safe shellscripts ~ https://sipb.mit.edu/doc/safe-shell/ https://sipb.mit.edu/doc/safe-shell/
- kps 10y agoThe big missing bashism in the list is the use of &> or >& to redirect both standard output and standard error. In a POSIX conforming shell, &> word is the same as & > word since '&>' is not a distinct token; it puts the command in the background and redirects standard output. The more commonly seen >& will bite you even in bash. What does this do? >& $FILENAME Answer: in bash, it depends on the spelling of the file name. Note that bash accepts both even in its so-called ‘POSIX compatible’ mode.
- zAy0LfpBZLC8mAC 10y agoThat temp file example is a vulnerable to symlink attacks.