4 ms·

If serious about creating shell script "apps", I think standard POSIX shell is the way to go. The Bashism is seldom needed, even though it is powerful. Why? Be

If serious about creating shell script "apps", I think standard POSIX shell is the way to go. The Bashism is seldom needed, even though it is powerful. Why? Because shell scripting is often used in DevOps, and containers very often do not come with Bash, but with (busybox) ash or dash instead. Say you are going FROM nginx to FROM nginx:alpine (to get that smaller image size), woha! your bash script is not working anymore.

CJefferson 6y ago
My main problem with POSIX shell is it is hard to test, as people might use bash, ash, zsh or dash as sh. Bash is at least not a moving target.
- saagarjha 6y ago
  You can run it through shellcheck, no?
- thomasbacklund 6y ago
  Using `shellcheck --shell sh` is a good start :) If adhering to POSIX it should run the same in bash, ash and dash. Bash is kind of a moving target in it self, since it comes with many new features. You have to be aware of which version you are targeting. For instance when using arrays, associative arrays, there is a diff between version 3 and 4. Version 5 is even more competent and it's easy start digging to deep into the cookie jar and getting your hand stuck in there :)
  - patrec 6y ago
    > Using `shellcheck --shell sh` is a good start :) > If adhering to POSIX it should run the same in bash, ash and dash. Not so. Shellcheck will tell you about syntactically invalid bashisms, but will not warn you about the zillions of things that are under or unspecified in posix. Pertinent minimal example: $ echo $'#!/bin/sh\ntrap "echo exiting" EXIT\nsleep 1h' > exit.sh $ shellcheck --shell sh exit.sh && echo "ALL GOOD!" $ /bin/dash exit.sh ^C $ /usr/bin/zsh exit.sh ^C $ /bin/bash exit.sh ^Cexiting Good luck finding a person who can use trap in a way that actually works as intended across different posix conform shells without consulting stack overflow first.
    - thomasbacklund 6y ago
      Indeed, it's such a messy job coding for shell.. Comes with a lot trial and error. The above will work as (tested in ash/dash/bash): #!/usr/bin/env sh trap "echo exiting" INT sleep 1h But to your point, yes, how to actually know that without doing trial n' error & stackoverflow. Thanks for the `sleep 1h`, didn't know that you could do hour as unit :) (edited: turned out INT is enough to trap)
      - patrec 6y ago
        Nope, INT is (generally) too specific :) Sometimes of course, you just want to run something on SIGINT and not on normal exit, SIGHUP or whatever. But maybe the main use case for trap is implementing try/finally logic and you really want it to fire whenever the shell exits, not matter how (to clean up resources like temporary files). Bash's `trap EXIT` mostly does what you want ("run this when the shell exits"), but writing something that behaves this way across shells (and doesn't fire multiple times etc). is amazingly painful.
        thomasbacklund 6y ago
        Ok, I though it was to catch CTRL-c :) trap EXIT, does work for me in ash/dash/bash, though, when just letting the code flow exit with and without errors. Not sure your use case, but I can for sure concur that traps are very painful to get right, especially when getting into child/grandchild processes land, taking terminal session, etc in regard.
- patrec 6y ago
  It would be nice if bash were not a moving target. I have run into several bash incompatibilities in real life. Bash on macOS differs substantially from bash on Linux because it's much older. But bash is also generally somewhat poorly designed, leading to lots of aggravating and puzzling behavior for "corner cases" and this behavior not infrequently changes between versions. So for example set -u for the longest time used to trigger on defined but empty arrays (which is hardly an obscure edge case), but what workarounds are possible differs quite a bit between versions: https://gist.github.com/dimo414/2fb052d230654cc0c25e9e41a9651ebe https://gist.github.com/dimo414/2fb052d230654cc0c25e9e41a965...
  - thomasbacklund 6y ago
    Yeah, I agree.. I was listening to the Command Line Heroes podcast the other day, about Bash. First version was stored on tape. So it has some legacy, hehe. Already then the requirement was to be backwards compatible with sh, so yeah, corner cases are around for sure. If writing bash I do aim at version 3, because as you say they got that on MacAttack.
rascul 6y ago
I'll target bash (version checks are easy enough if necessary) instead of whatever happens to be /bin/sh. It's no problem for me to install bash wherever I need it, it provides some features I use, and I don't have to concern myself with POSIX implementation differences. https://stackoverflow.com/questions/11376975/is-there-a-minimally-posix-2-compliant-shell https://stackoverflow.com/questions/11376975/is-there-a-mini...
- thomasbacklund 6y ago
  If you need some bash specific features, I'm not going to argue with that. In my experience, having portability of the script is quite nice when not necessarily controlling the environment it is going to run in, and aiming for ash/dash/bash does take it a long way.
patrec 6y ago
> The Bashism is seldom needed I wish that were true. But posix shell lacks two fairly vital features: sane trap and process redirection. In an ideal world the posix comittee would just fix that, but I won't hold my breath.
- thomasbacklund 6y ago
  It sounds like you need Bash then :) I totally agree Bash has some powerful features, some I go out of my way to reimplement in sh (as for example string operations [0]). But not all can ofc be replicated then bash it is. Maybe then: sh->bash->python/perl [0] https://gitlab.com/space-sh/string/-/blob/master/Spacefile.sh https://gitlab.com/space-sh/string/-/blob/master/Spacefile.s...
- bewuethr 6y ago
  I was going through the changes introduced in Bash 5.1, and to my surprise, it mentioned[1] that process substitution is now enabled in POSIX mode. I wondered if that means POSIX sh now supports it, but I haven't been able to find evidence for that, neither in the POSIX spec nor in the bug tracker of POSIX mailing list archives. Similarly, I remember reading about POSIX sh supporting arrays at some point in the future, but for the life of me, I can't find any evidence for this either. [1]: https://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES#n538 https://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES#n538
  - patrec 6y ago
    Arrays in shells generally suck (and behave quite different between different shells, e.g. zsh arrays are nothing like bash arrays), so I hope they don't make it into posix. OTOH, process substitution is a perfectly elegant and nice feature that fits well into the original design of bourne shell, removing an obvious limitation of pipes. It also behaves pretty much the same across shells that have it, so it should be much easier to standardize.

CJefferson 6y ago

My main problem with POSIX shell is it is hard to test, as people might use bash, ash, zsh or dash as sh. Bash is at least not a moving target.

saagarjha 6y ago
You can run it through shellcheck, no?
thomasbacklund 6y ago
Using `shellcheck --shell sh` is a good start :) If adhering to POSIX it should run the same in bash, ash and dash. Bash is kind of a moving target in it self, since it comes with many new features. You have to be aware of which version you are targeting. For instance when using arrays, associative arrays, there is a diff between version 3 and 4. Version 5 is even more competent and it's easy start digging to deep into the cookie jar and getting your hand stuck in there :)
- patrec 6y ago
  > Using `shellcheck --shell sh` is a good start :) > If adhering to POSIX it should run the same in bash, ash and dash. Not so. Shellcheck will tell you about syntactically invalid bashisms, but will not warn you about the zillions of things that are under or unspecified in posix. Pertinent minimal example: $ echo $'#!/bin/sh\ntrap "echo exiting" EXIT\nsleep 1h' > exit.sh $ shellcheck --shell sh exit.sh && echo "ALL GOOD!" $ /bin/dash exit.sh ^C $ /usr/bin/zsh exit.sh ^C $ /bin/bash exit.sh ^Cexiting Good luck finding a person who can use trap in a way that actually works as intended across different posix conform shells without consulting stack overflow first.
  - thomasbacklund 6y ago
    Indeed, it's such a messy job coding for shell.. Comes with a lot trial and error. The above will work as (tested in ash/dash/bash): #!/usr/bin/env sh trap "echo exiting" INT sleep 1h But to your point, yes, how to actually know that without doing trial n' error & stackoverflow. Thanks for the `sleep 1h`, didn't know that you could do hour as unit :) (edited: turned out INT is enough to trap)
    - patrec 6y ago
      Nope, INT is (generally) too specific :) Sometimes of course, you just want to run something on SIGINT and not on normal exit, SIGHUP or whatever. But maybe the main use case for trap is implementing try/finally logic and you really want it to fire whenever the shell exits, not matter how (to clean up resources like temporary files). Bash's `trap EXIT` mostly does what you want ("run this when the shell exits"), but writing something that behaves this way across shells (and doesn't fire multiple times etc). is amazingly painful.
      - thomasbacklund 6y ago
        Ok, I though it was to catch CTRL-c :) trap EXIT, does work for me in ash/dash/bash, though, when just letting the code flow exit with and without errors. Not sure your use case, but I can for sure concur that traps are very painful to get right, especially when getting into child/grandchild processes land, taking terminal session, etc in regard.
patrec 6y ago
It would be nice if bash were not a moving target. I have run into several bash incompatibilities in real life. Bash on macOS differs substantially from bash on Linux because it's much older. But bash is also generally somewhat poorly designed, leading to lots of aggravating and puzzling behavior for "corner cases" and this behavior not infrequently changes between versions. So for example set -u for the longest time used to trigger on defined but empty arrays (which is hardly an obscure edge case), but what workarounds are possible differs quite a bit between versions: https://gist.github.com/dimo414/2fb052d230654cc0c25e9e41a9651ebe https://gist.github.com/dimo414/2fb052d230654cc0c25e9e41a965...
- thomasbacklund 6y ago
  Yeah, I agree.. I was listening to the Command Line Heroes podcast the other day, about Bash. First version was stored on tape. So it has some legacy, hehe. Already then the requirement was to be backwards compatible with sh, so yeah, corner cases are around for sure. If writing bash I do aim at version 3, because as you say they got that on MacAttack.

rascul 6y ago

I'll target bash (version checks are easy enough if necessary) instead of whatever happens to be /bin/sh. It's no problem for me to install bash wherever I need it, it provides some features I use, and I don't have to concern myself with POSIX implementation differences. https://stackoverflow.com/questions/11376975/is-there-a-minimally-posix-2-compliant-shell https://stackoverflow.com/questions/11376975/is-there-a-mini...

thomasbacklund 6y ago
If you need some bash specific features, I'm not going to argue with that. In my experience, having portability of the script is quite nice when not necessarily controlling the environment it is going to run in, and aiming for ash/dash/bash does take it a long way.

patrec 6y ago

> The Bashism is seldom needed I wish that were true. But posix shell lacks two fairly vital features: sane trap and process redirection. In an ideal world the posix comittee would just fix that, but I won't hold my breath.

thomasbacklund 6y ago
It sounds like you need Bash then :) I totally agree Bash has some powerful features, some I go out of my way to reimplement in sh (as for example string operations [0]). But not all can ofc be replicated then bash it is. Maybe then: sh->bash->python/perl [0] https://gitlab.com/space-sh/string/-/blob/master/Spacefile.sh https://gitlab.com/space-sh/string/-/blob/master/Spacefile.s...
bewuethr 6y ago
I was going through the changes introduced in Bash 5.1, and to my surprise, it mentioned[1] that process substitution is now enabled in POSIX mode. I wondered if that means POSIX sh now supports it, but I haven't been able to find evidence for that, neither in the POSIX spec nor in the bug tracker of POSIX mailing list archives. Similarly, I remember reading about POSIX sh supporting arrays at some point in the future, but for the life of me, I can't find any evidence for this either. [1]: https://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES#n538 https://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES#n538
- patrec 6y ago
  Arrays in shells generally suck (and behave quite different between different shells, e.g. zsh arrays are nothing like bash arrays), so I hope they don't make it into posix. OTOH, process substitution is a perfectly elegant and nice feature that fits well into the original design of bourne shell, removing an obvious limitation of pipes. It also behaves pretty much the same across shells that have it, so it should be much easier to standardize.