4 ms·

> Regarding your argument against Bash, isn't it true that this is just a different version of the same bootstrapping problem, and that downloading a temporary

> Regarding your argument against Bash, isn't it true that this is just a different version of the same bootstrapping problem, and that downloading a temporary but specific Bash version would get around that? (I agree that working with Bash leaves a lot to be desired. But it's also more accessible to end-users to hack on, observe, and learn from, than a compiled Rust binary is. And tools like Copilot and ChatGPT and https://www.shellcheck.net/ https://www.shellcheck.net/ make working in Bash MUCH more painless, I've found.) I haven't been working on the detsys rust installer, but I think I'm the person who's contributed the most to the official bash multi-user installer since Graham did a lot of work on it several years ago. I ~like Shell, and I share your concern about it being less hackable. But the project has also been slowly bleeding goodwill around the lack of clean uninstall/reinstall for years, and I will make my peace with the compromises it takes to get there. The problem isn't so much conjuring good shell expressions. Nix leans a lot on Shell, and I feel like the community has a fair number of people around who know Shell fairly well. The problems are more like: - limited/tedious support for robust error-handling and recovery - needing a significant amount of new code to handle removing what Nix adds (even though I've done a lot of work to develop the patterns to support eventually getting here) - a fairly fat long tail of problems caused by things like edge-case environment/state differences (due to platform, due to users having surprising environments, etc.) and even flaky behavior on some systems - screwing up the installer is fairly high stakes (and it's very common for even one-line changes to break the installer in some way for someone somewhere and end up needing to get reverted for a rethink) - testing the installer is tedious (it's gotten a lot better over the past ~2 years--but that testing is also just covering the golden path), so iterating is slow - writing and reviewing installer changes tends to entail (and sometimes suffer from a lack of) really broad knowledge of different platforms, shells, utilities, options, permissions, and user configuration ~patterns Because the stakes are high and review is hard and testing is hard and iterating is slow, IME most of the people who are enthusiastic about working on it are naive about these headwinds. This might lead them to, say, submit a PR bigger than anyone will have the stomach to review/merge--or not have the time/energy to slog through weeks/months of review/edit/test iterations it takes to build high confidence in a big change. It's also a big drag on the number of people who have the stomach to continue making substantive installer PRs after they land one. Before detsys started looking at the rust installer, I had picked at a POC for just bundling a bash and everything we touch (iirc at least coreutils, diff, patch). If their effort failed, that's definitely the approach I was going to recommend next to reduce the frequency with which we get bitten by a utility quirk/change. But that still would've left quite a bit of work to do to support uninstall and idempotent reinstalls. I'm personally limiting the amount of non-fix work I do on the official installer to avoid falling into its gravity well and completely owning it. Once more, if any combination of the technical changes here are sufficient to multiply the number of people willing and able to touch the installer, I'll make my peace with the tradeoffs.

grhmc 4y ago
This is such a good summary of all the issues we've been running into over and over, and part of the deep motivations of our work here.
pxc 4y ago
> if any combination of the technical changes here are sufficient to multiply the number of people willing and able to touch the installer, I'll make my peace with the tradeoffs. Do you feel similarly about the Nix setup scripts that the installer works to ensure are sourced by user shells? I'd kinda love to see those go and be replaced by a little program that sets up a Nix environment and then launches a shell for you, kinda like the `login` program. The shell-based approach has made supporting incompatible shells kinda nasty and clunky.
- grhmc 4y ago
  I certainly feel that way.
- abathur 4y ago
  I have managed to avoid fiddling with the init hook scripts themselves, so I hadn't actually thought about this specific idea. IDK if my thoughts will be productive/incisive. I've definitely felt like the Nth-shell situation is untenable and that we'll either need to abstract a definition to generate the hooks from or find some other ~general/universal solution. I like the way this suggestion sounds with respect to getting out of that maintainability problem, but I'm not sure if it'll work quite right? Here's one crappy rabbit-hole we've gone down: Since macOS changed its default shell to zsh, they've been overwriting /etc/zshrc with every update (this evicts Nix's hook, confuses/scares people, and makes them try to reinstall; note that they aren't using the Relocated Items treatment we'd expect from how they do this with most stuff in /etc/ including /etc/bashrc while it was still the default shell). An enterprising zsh user PRed a fix that moves the hook to /etc/zshenv. macOS doesn't ship /etc/zshenv, so they don't overwrite it on update. It seems to work at a blush, but Nix and everything you install with Nix ends up at the end of your PATH instead of the beginning, so everything that depends on a PATH lookup will keep using your system utilities. Why? IIRC because the /etc/zshprofile macOS ships will do "eval `/usr/libexec/path_helper -s`", and path_helper will add things from /etc/paths and /etc/paths.d to the left of the existing PATH. I suspect if we inject a binary or even a script to preconfigure this environment, the call to path_helper in the zsh init process is still going to leave everything we added at the end of the PATH. We could in theory add an /etc/paths.d/nix, but path_helper appends this after everything in /etc/paths, so it has the same problem. We could stick our Nix-related directories at the top of /etc/paths, but macOS does ship this file so we have no reason to have faith that macOS updates don't overwrite it or won't start doing so at some point. (We might also be able to move this to /etc/zshenv and also define a `/usr/libexec/path_helper(){ ... }` function that behaves like WE want--but that could obviously have downstream effects on other code in user profile/init scripts that uses path_helper. No clue if this is common.)

grhmc 4y ago

This is such a good summary of all the issues we've been running into over and over, and part of the deep motivations of our work here.

pxc 4y ago

> if any combination of the technical changes here are sufficient to multiply the number of people willing and able to touch the installer, I'll make my peace with the tradeoffs. Do you feel similarly about the Nix setup scripts that the installer works to ensure are sourced by user shells? I'd kinda love to see those go and be replaced by a little program that sets up a Nix environment and then launches a shell for you, kinda like the `login` program. The shell-based approach has made supporting incompatible shells kinda nasty and clunky.

grhmc 4y ago
I certainly feel that way.
abathur 4y ago
I have managed to avoid fiddling with the init hook scripts themselves, so I hadn't actually thought about this specific idea. IDK if my thoughts will be productive/incisive. I've definitely felt like the Nth-shell situation is untenable and that we'll either need to abstract a definition to generate the hooks from or find some other ~general/universal solution. I like the way this suggestion sounds with respect to getting out of that maintainability problem, but I'm not sure if it'll work quite right? Here's one crappy rabbit-hole we've gone down: Since macOS changed its default shell to zsh, they've been overwriting /etc/zshrc with every update (this evicts Nix's hook, confuses/scares people, and makes them try to reinstall; note that they aren't using the Relocated Items treatment we'd expect from how they do this with most stuff in /etc/ including /etc/bashrc while it was still the default shell). An enterprising zsh user PRed a fix that moves the hook to /etc/zshenv. macOS doesn't ship /etc/zshenv, so they don't overwrite it on update. It seems to work at a blush, but Nix and everything you install with Nix ends up at the end of your PATH instead of the beginning, so everything that depends on a PATH lookup will keep using your system utilities. Why? IIRC because the /etc/zshprofile macOS ships will do "eval `/usr/libexec/path_helper -s`", and path_helper will add things from /etc/paths and /etc/paths.d to the left of the existing PATH. I suspect if we inject a binary or even a script to preconfigure this environment, the call to path_helper in the zsh init process is still going to leave everything we added at the end of the PATH. We could in theory add an /etc/paths.d/nix, but path_helper appends this after everything in /etc/paths, so it has the same problem. We could stick our Nix-related directories at the top of /etc/paths, but macOS does ship this file so we have no reason to have faith that macOS updates don't overwrite it or won't start doing so at some point. (We might also be able to move this to /etc/zshenv and also define a `/usr/libexec/path_helper(){ ... }` function that behaves like WE want--but that could obviously have downstream effects on other code in user profile/init scripts that uses path_helper. No clue if this is common.)