3 ms·

/usr/bin/env is required by POSIX, but bash's location already varies across distros and operating systems. NixOS's choices there are fundamentally correct.

by pxc 3y ago

charcircuit 3y ago
>/usr/bin/env is required by POSIX No, it isn't. The required directories are /, /dev, /tmp. The reqired files are /dev/null, /dev/tty, and /dev/console. https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ chapter 10 Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang (and they should stop opening PRs asking people to switch to /usr/bin/env for them) >but bash's location already varies across distros It almost always exists in both /bin/bash and /usr/bin/bash via a symlink. >NixOS's choices there are fundamentally correct I suspect the choice has more to do with that they don't want to do the work needed to remove /usr/bin/env.
- pxc 3y ago
  I thought the path was fixed by POSIX but I suppose it's only fixed by convention. Its availability is definitely part of POSIX, though: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/env.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/e... And one of its main uses in practice today is to perform lookup via PATH in shebangs, since it has to do that kind of lookup to be able to launch programs with modified environments as given in those examples anyhow. It obviously can't perform that common function if applications using it can't find it, so it's in an exceptional position relative to other utilities. NixOS just makes the correct tradeoff here for `env` to be able to perform one of its main functions by putting it in a conventional location. > Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang (and they should stop opening PRs asking people to switch to /usr/bin/env for them) Nixpkgs already has facilities to handle this for scripts that are packaged via Nix. Those shebangs matter only when (a) some impurity is actually desired or (b) the scripts are used/managed out-of-band. The issue also doesn't only affect NixOS. > It almost always exists in both /bin/bash and /usr/bin/bash via a symlink. What if I don't want to use the system bash (ever) because it's old? If someone isn't shipping their own bash or targeting a specific version that they know comes with a target platform very deliberately, why should they point to the system bash?
  - charcircuit 3y ago
    >NixOS just makes the correct tradeoff here for `env` to be able to perform one of its main functions by putting it in a conventional location. At that point you might as well do the same for sh and bash as it would also save a lot of unnecessary headache. Shebangs are not a necessary part of a distro. You can always call the interpreter directly and pass in the script. I personally find the idea of disrupting the normal FHS for purity to be a cool idea, but if you are going to ignore that goal and compromise to increase compatibility with existing scripts I feel you might as well compromise that purity more to support bash as that will solve the users of the OS even more problems. >What if I don't want to use the system bash (ever) because it's old? When you are sharing a bash script with others you typically aren't going to want to use bleeding edge features. I don't see that as a big problem. For your own scripts you can always use the full path to whatever version you want.
- abathur 3y ago
  > No, it isn't. The required directories are /, /dev, /tmp. The reqired files are /dev/null, /dev/tty, and /dev/console. To finish splitting the hair: the env utility is specified by POSIX (it also specifies sh, and not bash), but it doesn't specify the path of any standard utility. > Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang Weird hill. Shebangs are a "native" way to tell the ~system what interpreter to use, and optionally what arguments to pass to it. Can you clarify how this argument doesn't self-reduce to absurdity? Why should the distinction you're drawing apply to the "non-native" behavior of env and not to the non-native behavior of every other external interpreter and executable? > (and they should stop opening PRs asking people to switch to /usr/bin/env for them) Again--weird hill. I won't pretend the Nix community isn't a big source of these requests, but your sample may be skewed if you think Nix is the only source of these requests. For example, a narrow GH search (("/usr/bin/env bash" AND "/bin/bash" AND nix) -org:NixOS -org:nix-community) focused just on bash only gives me 429 issues and 283 PRs that smell nix-motivated (and some are references to "*NIX"). A similar search for macOS (("/usr/bin/env bash" AND "/bin/bash" AND macOS AND NOT nix AND NOT nixos) -org:NixOS -org:nix-community) turns up 1.8k issues and 1k PRs. > I suspect No real need to speculate--https://github.com/NixOS/nixpkgs/commit/7281a851b3594 https://github.com/NixOS/nixpkgs/commit/7281a851b3594 makes it somewhat clear that this is a convenience for end users that may want to use (or write) scripts ~can run on other NixOS and other distros. (There are definitely people in the community who'd be happy to remove both /bin/sh and /usr/bin/env. Both of these are even nullable options, though I don't think I've heard of them being used outside of the envfs module in nixpkgs and the FHS module in https://github.com/balsoft/nixos-fhs-compat https://github.com/balsoft/nixos-fhs-compat.) > they don't want to do the work needed to remove /usr/bin/env. Remove it from where? These are already extensively patched out in nixpkgs. If that's the work you're talking about, it gets done all the time. (This speculation might be more meaningful for /bin/sh; the comment it was introduced with in 2006 is still present: https://github.com/NixOS/nixpkgs/commit/b60dd36c8fccc5d04497385b8ed6f336a396bb87#diff-6387dac19d0ed1364fae32520c893101848a3123a86197cadb9708aa4de581d9R82-R85 https://github.com/NixOS/nixpkgs/commit/b60dd36c8fccc5d04497...)
  - charcircuit 3y ago
    >Can you clarify how this argument doesn't self-reduce to absurdity? I think you misunderstood me. My comment about being native is that Linux does not look at your path when handling the shebang. Since POSIX doesn't standardize where env is there is no way to know how to launch env ahead of time. If there is no way to write a generic shebang you might as well just accept that you will need to change it. >but your sample may be skewed if you think Nix is the only source of these requests. Fair point. I am also biased from PRs also trying to insert flake.nix into projects. My issue is that nix is doing something nonstandard and it feels like projects are having to accommodate nix's oddities instead of nix being the one to accommodate what already exists. >makes it somewhat clear that this is a convenience for end users that may want to use (or write) scriptsz Thanks for sharing this. I always assumed they had it from the beginning. >Remove it from where Not all instances of it in build scripts are properly rewritten. Also there would be work in creating a migration plan to not breaking people who currently rely on its existence.

/usr/bin/env is required by POSIX, but bash's location already varies across distros and operating systems. NixOS's choices there are fundamentally correct.

by pxc 3y ago

/usr/bin/env is required by POSIX, but bash's location already varies across distros and operating systems. NixOS's choices there are fundamentally correct.

charcircuit 3y ago

>/usr/bin/env is required by POSIX No, it isn't. The required directories are /, /dev, /tmp. The reqired files are /dev/null, /dev/tty, and /dev/console. https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ chapter 10 Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang (and they should stop opening PRs asking people to switch to /usr/bin/env for them) >but bash's location already varies across distros It almost always exists in both /bin/bash and /usr/bin/bash via a symlink. >NixOS's choices there are fundamentally correct I suspect the choice has more to do with that they don't want to do the work needed to remove /usr/bin/env.

pxc 3y ago
I thought the path was fixed by POSIX but I suppose it's only fixed by convention. Its availability is definitely part of POSIX, though: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/env.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/e... And one of its main uses in practice today is to perform lookup via PATH in shebangs, since it has to do that kind of lookup to be able to launch programs with modified environments as given in those examples anyhow. It obviously can't perform that common function if applications using it can't find it, so it's in an exceptional position relative to other utilities. NixOS just makes the correct tradeoff here for `env` to be able to perform one of its main functions by putting it in a conventional location. > Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang (and they should stop opening PRs asking people to switch to /usr/bin/env for them) Nixpkgs already has facilities to handle this for scripts that are packaged via Nix. Those shebangs matter only when (a) some impurity is actually desired or (b) the scripts are used/managed out-of-band. The issue also doesn't only affect NixOS. > It almost always exists in both /bin/bash and /usr/bin/bash via a symlink. What if I don't want to use the system bash (ever) because it's old? If someone isn't shipping their own bash or targeting a specific version that they know comes with a target platform very deliberately, why should they point to the system bash?
- charcircuit 3y ago
  >NixOS just makes the correct tradeoff here for `env` to be able to perform one of its main functions by putting it in a conventional location. At that point you might as well do the same for sh and bash as it would also save a lot of unnecessary headache. Shebangs are not a necessary part of a distro. You can always call the interpreter directly and pass in the script. I personally find the idea of disrupting the normal FHS for purity to be a cool idea, but if you are going to ignore that goal and compromise to increase compatibility with existing scripts I feel you might as well compromise that purity more to support bash as that will solve the users of the OS even more problems. >What if I don't want to use the system bash (ever) because it's old? When you are sharing a bash script with others you typically aren't going to want to use bleeding edge features. I don't see that as a big problem. For your own scripts you can always use the full path to whatever version you want.
abathur 3y ago
> No, it isn't. The required directories are /, /dev, /tmp. The reqired files are /dev/null, /dev/tty, and /dev/console. To finish splitting the hair: the env utility is specified by POSIX (it also specifies sh, and not bash), but it doesn't specify the path of any standard utility. > Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang Weird hill. Shebangs are a "native" way to tell the ~system what interpreter to use, and optionally what arguments to pass to it. Can you clarify how this argument doesn't self-reduce to absurdity? Why should the distinction you're drawing apply to the "non-native" behavior of env and not to the non-native behavior of every other external interpreter and executable? > (and they should stop opening PRs asking people to switch to /usr/bin/env for them) Again--weird hill. I won't pretend the Nix community isn't a big source of these requests, but your sample may be skewed if you think Nix is the only source of these requests. For example, a narrow GH search (("/usr/bin/env bash" AND "/bin/bash" AND nix) -org:NixOS -org:nix-community) focused just on bash only gives me 429 issues and 283 PRs that smell nix-motivated (and some are references to "*NIX"). A similar search for macOS (("/usr/bin/env bash" AND "/bin/bash" AND macOS AND NOT nix AND NOT nixos) -org:NixOS -org:nix-community) turns up 1.8k issues and 1k PRs. > I suspect No real need to speculate--https://github.com/NixOS/nixpkgs/commit/7281a851b3594 https://github.com/NixOS/nixpkgs/commit/7281a851b3594 makes it somewhat clear that this is a convenience for end users that may want to use (or write) scripts ~can run on other NixOS and other distros. (There are definitely people in the community who'd be happy to remove both /bin/sh and /usr/bin/env. Both of these are even nullable options, though I don't think I've heard of them being used outside of the envfs module in nixpkgs and the FHS module in https://github.com/balsoft/nixos-fhs-compat https://github.com/balsoft/nixos-fhs-compat.) > they don't want to do the work needed to remove /usr/bin/env. Remove it from where? These are already extensively patched out in nixpkgs. If that's the work you're talking about, it gets done all the time. (This speculation might be more meaningful for /bin/sh; the comment it was introduced with in 2006 is still present: https://github.com/NixOS/nixpkgs/commit/b60dd36c8fccc5d04497385b8ed6f336a396bb87#diff-6387dac19d0ed1364fae32520c893101848a3123a86197cadb9708aa4de581d9R82-R85 https://github.com/NixOS/nixpkgs/commit/b60dd36c8fccc5d04497...)
- charcircuit 3y ago
  >Can you clarify how this argument doesn't self-reduce to absurdity? I think you misunderstood me. My comment about being native is that Linux does not look at your path when handling the shebang. Since POSIX doesn't standardize where env is there is no way to know how to launch env ahead of time. If there is no way to write a generic shebang you might as well just accept that you will need to change it. >but your sample may be skewed if you think Nix is the only source of these requests. Fair point. I am also biased from PRs also trying to insert flake.nix into projects. My issue is that nix is doing something nonstandard and it feels like projects are having to accommodate nix's oddities instead of nix being the one to accommodate what already exists. >makes it somewhat clear that this is a convenience for end users that may want to use (or write) scriptsz Thanks for sharing this. I always assumed they had it from the beginning. >Remove it from where Not all instances of it in build scripts are properly rewritten. Also there would be work in creating a migration plan to not breaking people who currently rely on its existence.