6 ms·

Where is POSIX actually useful today? Is it mostly for shell scripts? Aren't people targetting bash or basic bourne shell features intead of posix? Is shellche

Where is POSIX actually useful today? Is it mostly for shell scripts? Aren't people targetting bash or basic bourne shell features intead of posix? Is shellcheck checking for best practices instead of POSIX compliance? And for other applications (GUI, servers, etc) strict POSIX compliance might be too restrictive? And with many things being Linux (or Linux-like like WSL) the need for this might be less? Are Android and/or iOS fully POSIX compliant? Any good blog or presentation describing the current state of POSIX?

susam 2y ago
> Aren't people targetting bash or basic bourne shell features intead of posix? I know many banks still have AIX systems with shells like ksh89, ksh93, etc. as the default shell. So if a shell script is written to work with a POSIX shell (instead of a particular shell), it has a better chance of running on such systems. Also, on Debian, the default non-interactive shell is dash [1]. This is the Debian Almquist Shell (dash). It is a POSIX-compliant shell derived from ash. So again, if we write system scripts for Debian and want it to run on Debian without any hassle, it makes sense to write the system scripts to conform to POSIX shell. Although shellcheck cannot perform full POSIX compliance check at this time, it is still a pretty good tool that can help with checking compliance with dash in particular. [1]: https://packages.debian.org/stable/dash https://packages.debian.org/stable/dash
- throw0101d 2y ago
  > So again, if we write system scripts for Debian and want it to run on Debian without any hassle, it makes sense to write the scripts to conform to POSIX shell. Or explicitly use bash in your shebang. One of the problems with Bash is that it insists on doing bash-y things even when you tell it to act like sh. People ask why you should write (or at least test) code to be multi-platform (even the basics of running it on BSD or macOS): it's because it forces you to be honest. Things change and initial assumptions may not be the same forever. * https://wiki.debian.org/Shell https://wiki.debian.org/Shell * https://archlinux.org/packages/?name=checkbashisms https://archlinux.org/packages/?name=checkbashisms * https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh
  - ko1nksm 2y ago
    > One of the problems with Bash is that it insists on doing bash-y things even when you tell it to act like sh. Its behaviour is a common behaviour of all sh, not just Bash.
- lye 2y ago
  Bash has `Priority: required` and is marked as "essential", it's available on every Debian system.
  - susam 2y ago
    Indeed, Bash is always available on Debian! After all, Debian uses Bash as the default interactive shell. But that's not the point of writing system scripts for dash. They are written for dash because that's still the default non-interactive shell. And it is so because dash is leaner and faster. Quoting from the link I posted in my previous comment: > Since it executes scripts faster than bash, and has fewer library dependencies (making it more robust against software or hardware failures), it is used as the default system shell on Debian systems.
kstrauser 2y ago
I’ve wondered the same. The most common standard I’ve seen in everyday work for a long time now is “runs on my Mac and the Linux server we’re deploying to”. I’m not talking about shops that ship software that customers receive and install on prem on their HPUX or whatever. That’s still a thing and people have to take that into account. I’m grateful I’m no longer among them.
jjmarr 2y ago
The original discussion on why Debian switched from bash to dash for /bin/sh is insightful. https://lwn.net/Articles/343924/ https://lwn.net/Articles/343924/ One big factor is for performance reasons in shell scripts. At the time, the switch decreased boot times for Debian by 7.5%. Bourne shell features add a lot of overhead and that's not always an acceptable tradeoff. Also, if you're using bash features in a script, you can always just add #!/bin/bash to the top of your file instead of #!/bin/sh to force a bash compatible shell.
- arp242 2y ago
  > One big factor is for performance reasons in shell scripts. At the time, the switch decreased boot times for Debian by 7.5%. Bourne shell features add a lot of overhead and that's not always an acceptable tradeoff. I seem to recall it was much smaller than that, something like 4% on a 2008 EEE-PC or something like that, but I can't find any numbers on that right now. The Debian startup scripts were already POSIX; it's not hard to get better performance out of zsh or bash by avoiding expensive processes lookups. Overall, I consider this to be mostly a myth, or at least extremely simplistic.
- cesarb 2y ago
  > One big factor is for performance reasons in shell scripts. At the time, the switch decreased boot times for Debian by 7.5%. And that should no longer be a relevant factor, since most of the boot process is now implemented directly in C (within systemd), instead of a bunch of shell scripts.
- _7tgr 2y ago
  Prefer `#!/usr/bin/env bash` instead, since /bin/bash isn't a standardized location for bash (even across Linux distros). The former causes $PATH to be searched for bash.
  - n_plus_1_acc 2y ago
    Is /usr/bin/env a standardized location?
throw0101d 2y ago
> Where is POSIX actually useful today? Defining a stable API to code against? > And with many things being Linux (or Linux-like like WSL) the need for this might be less? Define "being Linux". RHEL? Ubuntu? Other? Is /bin/sh linked to Bash or something? * https://mywiki.wooledge.org/Bashism https://mywiki.wooledge.org/Bashism * https://linux.die.net/man/1/checkbashisms https://linux.die.net/man/1/checkbashisms > Are Android and/or iOS fully POSIX compliant? UNIX® Certified Products include macOS: * https://www.opengroup.org/openbrand/register/ https://www.opengroup.org/openbrand/register/ POSIX: * https://posix.opengroup.org/register.html https://posix.opengroup.org/register.html
chuckadams 2y ago
It's as good as any other standard: as a baseline of agreed-upon "correct" behavior for interoperability. Anything you add from there is gravy. It's more for implementers than users, e.g. for someone writing a new shell rather than writing shell scripts. Having the standard handy is also pretty useful when writing foreign interfaces, like the posix modules in python and perl. As for the current state of POSIX, well, you're looking at it. Might find a blog or two of someone on the POSIX committees, but the organizations aren't the kind that keep blogs. Probably best to just dive into the Wikipedia article on POSIX and start following the references on the bottom. You'll probably want to look into SUS, the Single Unix Specification, as well: it's identical to POSIX (plus curses for some reason) but it's the label that OS vendors may use rather than POSIX. macOS and some Linux distributions claim to be fully SUS-compliant; Linux as a whole does not, because its official scope is limited to the kernel which only implements a subset of POSIX. Fun fact: the name "POSIX" was coined by Richard Stallman.
denvaar 2y ago
My view is that shell scripting feels like the wild-west, so I try to conform to POSIX to maintain some level of sanity, though it feels restrictive at times. I rely on ShellCheck to help me write shell scripts that are POSIX compliant.
znpy 2y ago
I’m probably going to be downvoted for this, but anyway: the single unix specification has always been the only place where i could find C headers documentation in by header file. Meaning: i don’t want to know all the 6000 functions glibc support, i want to know what will (for example) net/inet.h will bring to my code (ideally with documentation). For sonme reason that doesn’t seem to be a thing. Not for glibc for sure. But the SUS does that. And i like it.
skissane 2y ago
> Where is POSIX actually useful today? Every now and again, I get annoyed by those cases where Linux has some API/utility/etc but macOS doesn't. Getting that API/utility into POSIX greatly increases the odds that Apple will end up implementing it. (Whether just by copying it from FreeBSD, or by writing it themselves.)
PhilipRoman 2y ago
Embedded shell scripts typically end up targeting the common denominator of bash/BusyBox/coreutils but the POSIX standards are a pretty good reference too. People often don't realize this, but standards are useful even if they aren't implemented 100% fully and correctly.

susam 2y ago

> Aren't people targetting bash or basic bourne shell features intead of posix? I know many banks still have AIX systems with shells like ksh89, ksh93, etc. as the default shell. So if a shell script is written to work with a POSIX shell (instead of a particular shell), it has a better chance of running on such systems. Also, on Debian, the default non-interactive shell is dash [1]. This is the Debian Almquist Shell (dash). It is a POSIX-compliant shell derived from ash. So again, if we write system scripts for Debian and want it to run on Debian without any hassle, it makes sense to write the system scripts to conform to POSIX shell. Although shellcheck cannot perform full POSIX compliance check at this time, it is still a pretty good tool that can help with checking compliance with dash in particular. [1]: https://packages.debian.org/stable/dash https://packages.debian.org/stable/dash

throw0101d 2y ago
> So again, if we write system scripts for Debian and want it to run on Debian without any hassle, it makes sense to write the scripts to conform to POSIX shell. Or explicitly use bash in your shebang. One of the problems with Bash is that it insists on doing bash-y things even when you tell it to act like sh. People ask why you should write (or at least test) code to be multi-platform (even the basics of running it on BSD or macOS): it's because it forces you to be honest. Things change and initial assumptions may not be the same forever. * https://wiki.debian.org/Shell https://wiki.debian.org/Shell * https://archlinux.org/packages/?name=checkbashisms https://archlinux.org/packages/?name=checkbashisms * https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh
- ko1nksm 2y ago
  > One of the problems with Bash is that it insists on doing bash-y things even when you tell it to act like sh. Its behaviour is a common behaviour of all sh, not just Bash.
lye 2y ago
Bash has `Priority: required` and is marked as "essential", it's available on every Debian system.
- susam 2y ago
  Indeed, Bash is always available on Debian! After all, Debian uses Bash as the default interactive shell. But that's not the point of writing system scripts for dash. They are written for dash because that's still the default non-interactive shell. And it is so because dash is leaner and faster. Quoting from the link I posted in my previous comment: > Since it executes scripts faster than bash, and has fewer library dependencies (making it more robust against software or hardware failures), it is used as the default system shell on Debian systems.

kstrauser 2y ago

I’ve wondered the same. The most common standard I’ve seen in everyday work for a long time now is “runs on my Mac and the Linux server we’re deploying to”. I’m not talking about shops that ship software that customers receive and install on prem on their HPUX or whatever. That’s still a thing and people have to take that into account. I’m grateful I’m no longer among them.

jjmarr 2y ago

The original discussion on why Debian switched from bash to dash for /bin/sh is insightful. https://lwn.net/Articles/343924/ https://lwn.net/Articles/343924/ One big factor is for performance reasons in shell scripts. At the time, the switch decreased boot times for Debian by 7.5%. Bourne shell features add a lot of overhead and that's not always an acceptable tradeoff. Also, if you're using bash features in a script, you can always just add #!/bin/bash to the top of your file instead of #!/bin/sh to force a bash compatible shell.

arp242 2y ago
> One big factor is for performance reasons in shell scripts. At the time, the switch decreased boot times for Debian by 7.5%. Bourne shell features add a lot of overhead and that's not always an acceptable tradeoff. I seem to recall it was much smaller than that, something like 4% on a 2008 EEE-PC or something like that, but I can't find any numbers on that right now. The Debian startup scripts were already POSIX; it's not hard to get better performance out of zsh or bash by avoiding expensive processes lookups. Overall, I consider this to be mostly a myth, or at least extremely simplistic.
cesarb 2y ago
> One big factor is for performance reasons in shell scripts. At the time, the switch decreased boot times for Debian by 7.5%. And that should no longer be a relevant factor, since most of the boot process is now implemented directly in C (within systemd), instead of a bunch of shell scripts.
_7tgr 2y ago
Prefer `#!/usr/bin/env bash` instead, since /bin/bash isn't a standardized location for bash (even across Linux distros). The former causes $PATH to be searched for bash.
- n_plus_1_acc 2y ago
  Is /usr/bin/env a standardized location?

throw0101d 2y ago

> Where is POSIX actually useful today? Defining a stable API to code against? > And with many things being Linux (or Linux-like like WSL) the need for this might be less? Define "being Linux". RHEL? Ubuntu? Other? Is /bin/sh linked to Bash or something? * https://mywiki.wooledge.org/Bashism https://mywiki.wooledge.org/Bashism * https://linux.die.net/man/1/checkbashisms https://linux.die.net/man/1/checkbashisms > Are Android and/or iOS fully POSIX compliant? UNIX® Certified Products include macOS: * https://www.opengroup.org/openbrand/register/ https://www.opengroup.org/openbrand/register/ POSIX: * https://posix.opengroup.org/register.html https://posix.opengroup.org/register.html

chuckadams 2y ago

It's as good as any other standard: as a baseline of agreed-upon "correct" behavior for interoperability. Anything you add from there is gravy. It's more for implementers than users, e.g. for someone writing a new shell rather than writing shell scripts. Having the standard handy is also pretty useful when writing foreign interfaces, like the posix modules in python and perl. As for the current state of POSIX, well, you're looking at it. Might find a blog or two of someone on the POSIX committees, but the organizations aren't the kind that keep blogs. Probably best to just dive into the Wikipedia article on POSIX and start following the references on the bottom. You'll probably want to look into SUS, the Single Unix Specification, as well: it's identical to POSIX (plus curses for some reason) but it's the label that OS vendors may use rather than POSIX. macOS and some Linux distributions claim to be fully SUS-compliant; Linux as a whole does not, because its official scope is limited to the kernel which only implements a subset of POSIX. Fun fact: the name "POSIX" was coined by Richard Stallman.

denvaar 2y ago

My view is that shell scripting feels like the wild-west, so I try to conform to POSIX to maintain some level of sanity, though it feels restrictive at times. I rely on ShellCheck to help me write shell scripts that are POSIX compliant.

znpy 2y ago

I’m probably going to be downvoted for this, but anyway: the single unix specification has always been the only place where i could find C headers documentation in by header file. Meaning: i don’t want to know all the 6000 functions glibc support, i want to know what will (for example) net/inet.h will bring to my code (ideally with documentation). For sonme reason that doesn’t seem to be a thing. Not for glibc for sure. But the SUS does that. And i like it.

skissane 2y ago

> Where is POSIX actually useful today? Every now and again, I get annoyed by those cases where Linux has some API/utility/etc but macOS doesn't. Getting that API/utility into POSIX greatly increases the odds that Apple will end up implementing it. (Whether just by copying it from FreeBSD, or by writing it themselves.)

PhilipRoman 2y ago

Embedded shell scripts typically end up targeting the common denominator of bash/BusyBox/coreutils but the POSIX standards are a pretty good reference too. People often don't realize this, but standards are useful even if they aren't implemented 100% fully and correctly.