9 ms·
Pure sh bible – Posix sh alternatives to external processes
- klodolph 6y agoI appreciate that this isn’t full of Bash-isms—it’s sometimes hard to find out how to do something in a shell script, because you get a bunch of Bash results.
- nevertoolate 6y agowhy the bash hate? I thought it was really uncommon for any unixlike os to come without bash (sh symlinked to bash)
- aidenn0 6y agoI don't know if it still is, but for a decent while, Debian sh was not bash.
- matvore 6y agoThe `/bin/bash` on macOS is severely out of date. So a script that starts with `#!/bin/bash` will be unable to use a lot of features. If you only rely on `/bin/sh` the user has the flexibility to point `/bin/sh` to whatever shell on their system is fastest and most up-to-date (e.g. `/bin/dash` is a pretty good choice for performance).
- wott 6y agoWhy speak of hate? The main problem I have with bashisms is that most people who use them just assume everyone runs bash as default shell. Instead of specifying xxx/bin/bash in their script, they use xxx/bin/sh (and they don't run any check about which kind of shell is running to ensure that bash is running and fail gracefully otherwise, and they don't either document that the script must be run from bash). There's no problem about writing in a specific language, if you clearly say you did. If I stretch the example a bit, I wouldn't like a Python script to start with xxx/bin/ruby.
- josephcsible 6y agoIt's not bash hate. It's that a lot of the time, you don't have bash available (e.g., BusyBox environments), and you want your scripts to still work.
- eikenberry 6y agoJust because it is present, doesn't mean its the default or used internally by things that call shell scripts. For example... Not that long ago I started a new job where I took over as maintainer of a project with a fairly extensive Makefile, including targets for release. I cleaned it up some but missed a couple bashisms. The original author wrote it on a Mac, on which /bin/sh is bash, and used bashisms. I took over on Linux, which has /bin/sh as dash (POSIX). So while the author used it fine, it broke for me in subtle ways that I didn't notice until I made my first, broken release (error was in formatting of md5 signature file). The problem with bashisms is that people don't think through if their current use case is an appropriate one.
- unphased 6y agojust change the shebang
- dazzawazza 6y agoNone of my FreeBSD machines have bash installed because it's not part of FreeBSD.
- klodolph 6y agoThere is absolutely no Bash hate here. If you do an experiment you'll easily find that dash is faster than bash, and since dash basically just a bare POSIX shell with very few extensions, a script that runs in dash is more likely to run anywhere (ignoring the fact that nearly every program you call from a shell script comes in both a GNU and BSD variant, and possibly a more minimalist POSIX version). So I write my shell scripts to dash, which is /bin/sh on my system (Debian). If you use Bash-isms, you should definitely not put #!/bin/sh at the top of your script, you should be putting #!/bin/bash
- 1vuio0pswjnm7 6y agoIf someone asked you to list all the UNIX-like OS you know about, would BSD or Plan9 make the list.
- patrec 6y agoLinux in practice always comes with bash, but sh is not necessarily an alias; e.g. on the world's most popular distro: > readlink -f /bin/sh /usr/bin/dash Also, BSDs and derivatives either have no bash installed by default (e.g. OpenBSD) or, in the case of macOS a horribly outdated one. Finally there are good reasons to hate bash: Bourne shell is flawed but brilliant, whereas bash is a bloated mess of unpredictable and counter-intuitive behaviour that also tends to change between versions. It's also much, much slower than e.g. dash. Unfortunately even for pure shell scripting bash in practice is the way to go; posix shell (and dash) unfortunately lack a few things that are pretty crucial and very cumbersome to work around, like process substitution.
- CDRdude 6y ago> or, in the case of macOS a horribly outdated one. The latest version of macOS defaults to zsh instead of the outdated version of bash.
- patrec 6y agoThat's mostly irrelevant, since basically no one writes shell code in zsh and this is unlikely to change that.
- ByteJockey 6y agoThat's probably going to change with macs having it as the default. Never underestimate the power of "Well, it worked on my machine" People seem to be willing to do apt-gets (or equivalent) in docker files, I could see several of my old co-workers doing that just to get a script working.
- patrec 6y agoUnlikely. Apple have made the default interactive shell zsh, but /bin/bash still stays around and I believe /bin/sh still aliases to it by default. Meanwhile there is an enormous ecosystems of shell scripts including countless curl ... | bash installers which no one has any motivation to port to some shell which is not installed by default on most Linux distros. Although Microsoft would undoubtedly be delighted with Apple accelerating the move of developers to Windows as the Unix of choice, I'm not convinced even Apple's current management is going to be dumb enough to remove bash altogether.
- patrec 6y agoActually, there are still some bashism, and real foot traps at that. `trap EXIT` basically only "works" for try/finally behavior in bash, not even zsh. There should really be a warning around this in the article: if you want to be have portable cleanup for all exit conditions that works in different shells, you need a lot of busywork.
- quotemstr 6y agoAs you should. Bash --- at least bash 3.x --- is available literally everywhere and has many features essential for robust programming, like local variables. Instead of writing for some antique shell, we should all just write for bash or zsh or something modern. I don't care about being compatible with some random AIX installation that's from 1870 and powered by a steam engine.
- jgtrosh 6y agoIt is however useful to know the difference, and in some cases you actually need to write POSIX (code meant to be sourced for example).
- umanwizard 6y agoOne counterexample: bash isn’t available in the base system of typical BSD variants (though, yes, you can install it from ports).
- trasz 6y agoThis would make sense if it said "zsh" instead. Not only is only better in terms of features, it's also released under a non-problematic Open Source license.
- klodolph 6y agoIf I want to use modern features, I’ll just use Perl or Python, both of which are also available literally everywhere.
- dimitrios1 6y agoOr I'll just write it in Go and easily cross compile an execuable binary for whatever platform I need.
- saagarjha 6y agosh is used because it is POSIX sh–you know it will work not "literally everywhere" but really, truly, literally everywhere. And your bashisms aren't going to do all that well on BusyBox, or dash. Just because you don't care doesn't mean that we should make incompatible scripts and not be aware that we are doing so.
- ljm 6y agoThis is a nice resource. Many moons ago when I first heard of Kakoune [1], I wanted to write some of my own plugins for it. Whether or not the philosophy has changed, back then it was 'just use shell scripts and make sure they're POSIX compliant'. That's when I learned about things like named pipes, the `mkfifo` command, and that it does take quite a lot of conscious effort to not accidentally include a convenient bashism. That said, there was nothing stopping you from writing the main functionality in another language and just shelling out to it. No need for the editor's config language to include primitives to do most of what a programming language would give you. [1] http://kakoune.org/ http://kakoune.org/
- chasil 6y agoThe first Korn shell from 1988 far predates the POSIX shell, and is a much richer language (from which BASH has stolen very much). As I understand it, ksh88 had to run on a 286 processor with 64kb data and instruction (segmented memory). The code required to do this was byzantine and frightening to maintain. Because all of these features could not be included in maintainable code on systems with very low memory, the POSIX shell eviscerated the ksh88 language standard. Surprisingly, checking the HP-UX man page (man sh-posix) gives most ksh88 functionality, which is definitly not in the Almquist shell (AFAIK the most popular POSIX shell). It is unfortunate that the POSIX shell had to take such a large step backwards when a far more powerful language predated it, but the reasoning is clear (and mostly centered on Xenix on a 286).
- jgtrosh 6y agoIsn't dash the more popular/common POSIX shell nowadays? Not sure how to go about evaluating this claim.
- chasil 6y agoDash is the Almquist shell. Most shells needed updates to fully-conform with the POSIX standard. The wiki mentions that the Almquist shell did not implement a "test" program that fully conformed. The sh-posix on HP-UX also had subtle changes to the ksh88 source to bring it into compliance (but this shell still supported arrays and coprocesses). https://en.wikipedia.org/wiki/Almquist_shell https://en.wikipedia.org/wiki/Almquist_shell
- adrianmonk 6y agoOn the subject of optimizing tput, this article mentions using hard-coded strings. Another approach (for some tput commands) is to run the command once, save its output in a variable, and re-use it. For example, a slow version of printing a blank chess board: #! /bin/sh for rowpair in 1 2 3 4 do for colpair in 1 2 3 4 do echo -n " $(tput rev) $(tput sgr0)" done echo for colpair in 1 2 3 4 do echo -n "$(tput rev) $(tput sgr0) " done echo done And a faster version: #! /bin/sh rev=$(tput rev) sgr0=$(tput sgr0) for rowpair in 1 2 3 4 do for colpair in 1 2 3 4 do echo -n " $rev $sgr0" done echo for colpair in 1 2 3 4 do echo -n "$rev $sgr0 " done echo done In many cases, tput is doing doing anything more than looking up a string and printing it. Though in other cases like "tput cols", it is doing more. (And anyway, the number of columns isn't a constant.)
- emmelaich 6y agoAnd you can of course put the whole chessboard in a string ... blackwhite=$(tput rev; echo -n " "; tput sgr0; echo -n " ") whiteblack=$(tput sgr0; echo -n " "; tput rev; echo -n " ") row_odd=$(for col in {1..4}; do echo -n "$whiteblack"; done; echo) row_even=$(for col in {1..4}; do echo -n "$blackwhite"; done; echo) chessboard=$(for i in {1..4}; do echo "$row_odd"; echo "$row_even"; done;tput sgr0)
- JdeBP 6y agoAs the headlined page is about portable shell script coding, note that the -n option to the echo utility is not portable. You should be using printf here, to be in the spirit of the headlined page. (And you should also be quoting variable assignments for safety.) There's a long history to this, which starts with the fact that adding any option support at all to the original echo breaks stuff. The result over the years has been quite a mess. The current Single Unix Specification has an note stating outright that one should simply not use -n or escape sequences at all with echo if one desires portability, and also stating that one should use printf instead. * https://unix.stackexchange.com/q/65803/5132 https://unix.stackexchange.com/q/65803/5132 * https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/e...
- emmelaich 6y agoThis uses e.g. i=$(($i + 1)) to increment. The simpler and possibly faster way is just ((i++)) or ((i+=1)) This syntax works in macos ksh, so pretty sure it's POSIX. Also, ksh93 supports sequence generation.. so as an alternative to the loops, use start=0 end=10 for i in {$start..$end}; do echo $i; done
- fratajcz 6y agoI don't think it's actually specified by POSIX, it only mentions arithmetic expansion with $(()) and there is no mention of (()) for arithmetic operations in the grammar: https://pubs.opengroup.org/onlinepubs/007904875/utilities/xcu_chap02.html#tag_02_10 https://pubs.opengroup.org/onlinepubs/007904875/utilities/xc... The example given for arithmetic expansion also uses x=$(($x-1)): https://pubs.opengroup.org/onlinepubs/007904875/utilities/xcu_chap02.html#tag_02_06_04 https://pubs.opengroup.org/onlinepubs/007904875/utilities/xc...
- emmelaich 6y agoInteresting! I wonder if any shells that support $((..)) do not support ((..) though. The update 2017 standard mentions ((..)) but not in the obvious place. It's under 'compound commands/grouping' ... https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_04 https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... > If a character sequence beginning with "((" would be parsed by the shell as an arithmetic expansion if preceded by a '$', shells which implement an extension whereby "((expression))" is evaluated as an arithmetic expression may treat the "((" as introducing as an arithmetic evaluation instead of a grouping command. A conforming application shall ensure that it separates the two leading '(' characters with white space to prevent the shell from performing an arithmetic evaluation.
- Crestwave 6y agoShells such as dash and ash do not support it, which are the most common test cases for POSIX scripts. It should also be noted that pre-increment itself is considered optional by POSIX, although += should work.
- emmelaich 6y agoGeneral comment ... I'm hesitant to fiddle with IFS; if the function exits early it can make the rest of the script nonsensical. Same goes for other 'globals' or externals such as globbing and using tput. You have to be careful to trap errors and return things to sanity.
- cryptonector 6y agoUse a subshell.
- ak217 6y agoI wish Bash had proper support for in-process subshells. I timed my code recently, and a variety of loop constructs was dramatically faster without subshells - but often required crazy contortions to avoid spawning one.
- senorsmile 6y agothey also have a pure bash bible: https://github.com/dylanaraps/pure-bash-bible https://github.com/dylanaraps/pure-bash-bible