10 ms·

Nix Survival Mode: macOS upgrades won't break Nix anymore

117 points by grhmc 3y ago

slekker 3y ago
Perhaps a stupid question but I got an early-March installation, can I just run the command in the blog post to upgrade it?
- biggestlou 3y ago
  Unfortunately, you will need to uninstall Nix[1] and then re-install it using that script. But given how we've built the installer that should be pretty seamless. 1. https://github.com/determinateSystems/nix-installer#uninstalling https://github.com/determinateSystems/nix-installer#uninstal...
  - grhmc 3y ago
    Yeah, this is unfortunate. We've built the Determinate Nix Installer to be super reliable and safe. We've built the receipt mechanism to have very clear and safe "revert" phases. This has made it hard to architect an upgrade of the installer's work itself ... but I'll put a ticket it! :)
  - jolux 3y ago
    I just uninstalled and reinstalled and it gave me 0.13.1 and not 0.14. Did I do something wrong? edit: tried again and got 0.14. Weird.
    - flexiondotorg 3y ago
      Sorry about that. Just checked. This is due to our phased rollouts which are currently 50/50. We're just ramping 0.14 up to 100%. So please try again in about 10 mins
      - deleted 3y ago
        [deleted]
      - grhmc 3y ago
        This is just rolled out now! Sorry about that, I thought we'd hit merge on that PR before publishing :').
behnamoh 3y ago
Can someone explain what the advantage of Nix is over Homebrew? I hear some stories where brew broke something, but I've never had any issues with it.
- unshavedyak 3y ago
  For me it's that my dotfiles and packages are consistent across all my envs. 2 Linux machines, 2 Mac machines. I'm not sure if Homebrew has this - but it takes me a few seconds to spin up a new Mac entirely with everything i want (minus build times lol). With home brew i was fiddling with "oh yea, this package/lib/plugin is missing" for weeks. edit: Also, all repos i have are also consistent. Ie i build my repos with specific versions of npm, rust, etc - all per repo, all consistent across all machines, all managed by Nix. Think node-env or py-env (or whatever they're called), but Nix and cross-language instead of specific to node or python ecosystems.
  - nani8ot 3y ago
    Using brew on Linux is generally not a good idea because distros have better package managers.
  - mikemcquaid 3y ago
    Homebrew has this. Check out Brewfiles and homebrew-bundle. If you want a (excuse self-promotion) dedicated but simple tool to bootstrap new systems: check out https://github.com/MikeMcQuaid/strap https://github.com/MikeMcQuaid/strap
- 0xCMP 3y ago
  While using `nix-env` or `nix profile` (the new way) are very homebrew-like a huge reason to use Nix is that it is more than a package manager. Things like nix-darwin and home-manager allow you to decoratively define your system config and dotfiles. Because it is a language you can override things in same place you're defining what packages or etc you want. And if you start using `direnv` with it's flakes plugin (enabled by default when installed using home-manager!) you can define a flake.nix which will keep a lock file independent of your system's/configuration's that means anyone with Nix installed should be able to activate a native dev environment with the exact versions of node, python, rust, pnpm, etc. ready to go. It's quite powerful once you become familiar with how to use it.
Cu3PO42 3y ago
Which breakages does this fix exactly? The fact that macOS upgrades may overwrite /etc/zshrc, etc? Other than that I haven't encountered any breakages whatsoever.
- biggestlou 3y ago
  Your hunch is exactly right. macOS upgrades typically do precisely that and it's quite annoying. I work at Determinate Systems as a full-time macOS user and it's one of those things I kind of quietly tolerated and didn't give much thought, but having this fixed removes a real headache.
  - Cu3PO42 3y ago
    Right! It's great to see that tackled for potentially all Nix installations! I have a workaround for that particular issue in my nix-darwin and Home-Manager configurations, but I've always disliked that I needed one in the first place. While we're speaking of things I need workarounds for: having my login shell be Nix-installed currently has a race condition on systems with FileVault. I have, for example, iTerm2 configured to resume on reboot and it will start before the Nix partition is mounted, thus complaining that it cannot exec my shell. My workaround entails placing a thin wrapper on the main partition that simply waits for /nix to be mounted and then `exec`ing the right shell binary depending on the invoking user. I should probably open an issue about this, but I'm not sure it can be solved at all while `/nix` needs to be on a seperate volume.
    - grhmc 3y ago
      Hmmm... would you mind opening an issue over here[1]? We have seen some related issues that might dovetail nicely. 1. https://github.com/determinateSystems/nix-installer https://github.com/determinateSystems/nix-installer
      - Cu3PO42 3y ago
        I will do tomorrow! In the meantime, on the offchance anyone stumbles across this who has the same problem, my workaround is available here [0]. (Also exported as `darwinModules.chsh` from my Flake) [0] https://github.com/Cu3PO42/gleaming-glacier/blob/master/modules/darwin/chsh/default.nix https://github.com/Cu3PO42/gleaming-glacier/blob/master/modu...
georgelyon 3y ago
It seems kind of wild that "installing some binaries on my system" requires these sorts of crazy workarounds like Launch Deamons. I'd personally love something like nix that would live nicely in a low-privilege folder somewhere in my user directory, just to install the few tools I need for a particular project. Currently I can achieve this with VSCode devcontainers and Docker low-privilege install. For system-wide tools I just download prebuilt binaries...
- Cu3PO42 3y ago
  I agree, the lengths the Nix installer currently goes to really shouldn't be required. However, this particular issue isn't about binaries being deleted. The Nix installer sets up some environment variables for you (in particular includes its binaries in PATH). For a system wide installation, it modifies /etc/zshrc for lack of a better option. Now, macOS likes to restore /etc/zshrc to its default on upgrades, thus breaking the environment variables. IMO this is no-one's fault except macOS.
  - pxc 3y ago
    The issue is trivial to diagnose and resolve if you understand the basics of shell startup and how Nix relies on bog-standard Unix features like PATH. But it's a real annoyance, and the kind of thing that could turn off or confuse new and casual users. And workarounds like this are not unusual on macOS, where upgrades verge on developer-hostile and user-expected behaviors are often impossible without hacks.
    - josephcsible 3y ago
      You mean it's trivial to work around. Only Apple can truly resolve it.
      - pxc 3y ago
        Right, it's trivial to restore the mangled configuration files, but permanently resolving the issue of OS upgrades blasting away user configuration files is on Apple.
pxc 3y ago
Drastic measures, but a vital workaround for a platform that is crucial in the wider development community. I love the Determinate Nix Installer! My interactions with the maintainer have been very positive, too. Can't recommend it enough. :)
- grhmc 3y ago
  Thank you a lot, pxc! I'm really glad you like it.
- flexiondotorg 3y ago
  Thanks for the kind words, I'll pass them along to Ana
amelius 3y ago
The fact that you need a third-party installer for Nix or otherwise it will break doesn't inspire much confidence in the whole setup.
- emptysongglass 3y ago
  It's macOS at fault here.
- grhmc 3y ago
  We wrote the Determinate Nix Installer with hopes that the greater project will adopt it[1]. Nix itself is fantastic. Improving the upstream Nix installer has been described as "unreasonably expensive and pointless"[2]. Hopefully it is adopted soon! 1. https://determinate.systems/posts/determinate-nix-installer https://determinate.systems/posts/determinate-nix-installer 2. https://discourse.nixos.org/t/nix-survival-mode-macos-upgrades-wont-break-nix-anymore-when-you-install-nix-using-the-determinate-nix-installer/34593/2?u=grahamc https://discourse.nixos.org/t/nix-survival-mode-macos-upgrad...
- whateveracct 3y ago
  Nix is a very bazaar language culture.
  - pxc 3y ago
    This! At any given time, a lot of Nix goodies are being prototyped outside of the Nix and Nixpkgs codebases. So it is with even the installer for Nix itself. But this is really mainly a cultural thing.
- deleted 3y ago
  [deleted]
oldbbsnickname 3y ago
I'd switch to nix package management on macOS if it were completely isolated to a single user, didn't demand use of sudo, creating /nix or other global changes, or running anything as root. Unfortunately, like Homebrew (except with some changes), MacPorts, and Fink, they seem to be set on global system changes that don't scale.
- grhmc 3y ago
  Fair enough! Unfortunately that isn't likely to be true any time soon, since macOS doesn't offer the namespacing primitives necessary for that to be possible like it is on Linux. You can bet once it is feasibly a good experience, DetSys will be hot on its tracks to make it happen. Until then, then :).
  - oldbbsnickname 3y ago
    Baloney. chroot and execvpe work with mere temporary directories. This is roughly how homebrew does "isolated" builds. You don't need virtualization, but you may want it.
    - grhmc 3y ago
      We don't need virtualization (that'd be easy.) We'd need mount namespacing and bind mounts to place a directory in your home directory at /nix during the process execution. EDIT: User and process namespacing would help a lot as well, to improve build time isolation.
      - oldbbsnickname 3y ago
        You're providing a synthetic filesystem to every process? If so, that sounds absurd and over-engineered.
        grhmc 3y ago
        No, however because of the general requirement to use /nix/store as a prefix we do support using this method for avoiding global installations.
russelg 3y ago
What do I need to do to switch from an official Nix install to using nix-installer on macOS? Just uninstall the current nix and install using nix-installer? My config is loosely based off https://github.com/dustinlyons/nixos-config https://github.com/dustinlyons/nixos-config if this matters. Using nix-darwin.
zaps 3y ago
Nix seems great for terminal-based development environments where starting a nix-shell sets up all the proper symlinks, env vars, etc. But how does this translate to IDEs like VSCode or IntelliJ? Like if you clone a repo with a shell.nix file, how does the IDE know to use that environment when you, say, use the File > Debug command, instead of executing whatever toolchain that happens to be on your path? Are there plugins available that enable this? I realize you could just open a terminal within the IDE, start a nix shell, and run the equivalent run/debug/etc commands, but I am curious if anything exists to make the experience seamless to the user.
- 1una 3y ago
  > Are there plugins available that enable this? Try https://github.com/direnv/direnv-vscode https://github.com/direnv/direnv-vscode or https://github.com/fehnomenal/intellij-direnv https://github.com/fehnomenal/intellij-direnv
- nextos 3y ago
  Yes, most Nix users employ https://direnv.net https://direnv.net or the equivalent for your IDE of choice. Emacs for instance has https://github.com/purcell/envrc https://github.com/purcell/envrc which set per-buffer variables. More background in https://github.com/direnv/direnv/wiki/Nix https://github.com/direnv/direnv/wiki/Nix.
  - rgoulter 3y ago
    That said, I believe VSCode (and some IDEs) don't necessarily support direnv in all cases.. e.g. you'd want direnv to load before other plugins (and update the PATH), but might not get that.
    - angio 3y ago
      If you start vscode with `code .` after entering the nix shell it will work in most cases.
- tripdout 3y ago
yubiox 3y ago
Can you use regular cpanm to install perl modules in nix perl?
eviks 3y ago
Have read many stories re. how complicated the official Nix configuration language is, so was wondering whether there is a plan for an easier config language (if not official, maybe a language that could be transpiled to the official one) similar to how this Rust-based installer is a big progress vs. the clunky bash?
- chpatrick 3y ago
  It's really not that complicated if you know functional languages. If anything it's quite minimal. It just looks a bit weird because it uses semicolons unlike most functional languages. I kind of wish they just kept JSON/Python syntax for attribute sets and lists. What's complicated (and often not or only lightly documented) is nixpkgs, the package library.
  - 0x6c6f6c 3y ago
    To that effect, https://mynixos.com https://mynixos.com has been a really awesome resource for docs. But yeah, the Nix language is definitely not the complicated part. You can get the language down in a day or two with their reference doc. What is difficult is understanding the patterns to NixOS and nixpkg's. Overlays, derivations, packages, flakes.. that's where things get the most confusing. It's a huge step away from other packaging mechanisms like PKGBUILDs but much more powerful and type-safe. But some things just aren't thought of, like home-managers `home.sessionPath` for updating the PATH is _append-only_.. so it literally cannot be used to prepend a preferred bin path. So you follow the typical convention as you might already and include it in the shells rc with `export PATH=/custom/path/bin:$PATH`. Which is the exact kind of safety you'd have today, so it's not worse, it's just another one of the shortcoming of the modules, as opposed to the Nic language itself. The same shortcoming would exist if you used HCL to declaratively define additional PATH paths. It's typically the module design, not the language.

117 points by grhmc 3y ago

Perhaps a stupid question but I got an early-March installation, can I just run the command in the blog post to upgrade it?

biggestlou 3y ago
Unfortunately, you will need to uninstall Nix[1] and then re-install it using that script. But given how we've built the installer that should be pretty seamless. 1. https://github.com/determinateSystems/nix-installer#uninstalling https://github.com/determinateSystems/nix-installer#uninstal...
- grhmc 3y ago
  Yeah, this is unfortunate. We've built the Determinate Nix Installer to be super reliable and safe. We've built the receipt mechanism to have very clear and safe "revert" phases. This has made it hard to architect an upgrade of the installer's work itself ... but I'll put a ticket it! :)
- jolux 3y ago
  I just uninstalled and reinstalled and it gave me 0.13.1 and not 0.14. Did I do something wrong? edit: tried again and got 0.14. Weird.
  - flexiondotorg 3y ago
    Sorry about that. Just checked. This is due to our phased rollouts which are currently 50/50. We're just ramping 0.14 up to 100%. So please try again in about 10 mins
    - deleted 3y ago
      [deleted]
    - grhmc 3y ago
      This is just rolled out now! Sorry about that, I thought we'd hit merge on that PR before publishing :').

behnamoh 3y ago

Can someone explain what the advantage of Nix is over Homebrew? I hear some stories where brew broke something, but I've never had any issues with it.

unshavedyak 3y ago
For me it's that my dotfiles and packages are consistent across all my envs. 2 Linux machines, 2 Mac machines. I'm not sure if Homebrew has this - but it takes me a few seconds to spin up a new Mac entirely with everything i want (minus build times lol). With home brew i was fiddling with "oh yea, this package/lib/plugin is missing" for weeks. edit: Also, all repos i have are also consistent. Ie i build my repos with specific versions of npm, rust, etc - all per repo, all consistent across all machines, all managed by Nix. Think node-env or py-env (or whatever they're called), but Nix and cross-language instead of specific to node or python ecosystems.
- nani8ot 3y ago
  Using brew on Linux is generally not a good idea because distros have better package managers.
- mikemcquaid 3y ago
  Homebrew has this. Check out Brewfiles and homebrew-bundle. If you want a (excuse self-promotion) dedicated but simple tool to bootstrap new systems: check out https://github.com/MikeMcQuaid/strap https://github.com/MikeMcQuaid/strap
0xCMP 3y ago
While using `nix-env` or `nix profile` (the new way) are very homebrew-like a huge reason to use Nix is that it is more than a package manager. Things like nix-darwin and home-manager allow you to decoratively define your system config and dotfiles. Because it is a language you can override things in same place you're defining what packages or etc you want. And if you start using `direnv` with it's flakes plugin (enabled by default when installed using home-manager!) you can define a flake.nix which will keep a lock file independent of your system's/configuration's that means anyone with Nix installed should be able to activate a native dev environment with the exact versions of node, python, rust, pnpm, etc. ready to go. It's quite powerful once you become familiar with how to use it.

Cu3PO42 3y ago

Which breakages does this fix exactly? The fact that macOS upgrades may overwrite /etc/zshrc, etc? Other than that I haven't encountered any breakages whatsoever.

biggestlou 3y ago
Your hunch is exactly right. macOS upgrades typically do precisely that and it's quite annoying. I work at Determinate Systems as a full-time macOS user and it's one of those things I kind of quietly tolerated and didn't give much thought, but having this fixed removes a real headache.
- Cu3PO42 3y ago
  Right! It's great to see that tackled for potentially all Nix installations! I have a workaround for that particular issue in my nix-darwin and Home-Manager configurations, but I've always disliked that I needed one in the first place. While we're speaking of things I need workarounds for: having my login shell be Nix-installed currently has a race condition on systems with FileVault. I have, for example, iTerm2 configured to resume on reboot and it will start before the Nix partition is mounted, thus complaining that it cannot exec my shell. My workaround entails placing a thin wrapper on the main partition that simply waits for /nix to be mounted and then `exec`ing the right shell binary depending on the invoking user. I should probably open an issue about this, but I'm not sure it can be solved at all while `/nix` needs to be on a seperate volume.
  - grhmc 3y ago
    Hmmm... would you mind opening an issue over here[1]? We have seen some related issues that might dovetail nicely. 1. https://github.com/determinateSystems/nix-installer https://github.com/determinateSystems/nix-installer
    - Cu3PO42 3y ago
      I will do tomorrow! In the meantime, on the offchance anyone stumbles across this who has the same problem, my workaround is available here [0]. (Also exported as `darwinModules.chsh` from my Flake) [0] https://github.com/Cu3PO42/gleaming-glacier/blob/master/modules/darwin/chsh/default.nix https://github.com/Cu3PO42/gleaming-glacier/blob/master/modu...

georgelyon 3y ago

It seems kind of wild that "installing some binaries on my system" requires these sorts of crazy workarounds like Launch Deamons. I'd personally love something like nix that would live nicely in a low-privilege folder somewhere in my user directory, just to install the few tools I need for a particular project. Currently I can achieve this with VSCode devcontainers and Docker low-privilege install. For system-wide tools I just download prebuilt binaries...

Cu3PO42 3y ago
I agree, the lengths the Nix installer currently goes to really shouldn't be required. However, this particular issue isn't about binaries being deleted. The Nix installer sets up some environment variables for you (in particular includes its binaries in PATH). For a system wide installation, it modifies /etc/zshrc for lack of a better option. Now, macOS likes to restore /etc/zshrc to its default on upgrades, thus breaking the environment variables. IMO this is no-one's fault except macOS.
- pxc 3y ago
  The issue is trivial to diagnose and resolve if you understand the basics of shell startup and how Nix relies on bog-standard Unix features like PATH. But it's a real annoyance, and the kind of thing that could turn off or confuse new and casual users. And workarounds like this are not unusual on macOS, where upgrades verge on developer-hostile and user-expected behaviors are often impossible without hacks.
  - josephcsible 3y ago
    You mean it's trivial to work around. Only Apple can truly resolve it.
    - pxc 3y ago
      Right, it's trivial to restore the mangled configuration files, but permanently resolving the issue of OS upgrades blasting away user configuration files is on Apple.

pxc 3y ago

Drastic measures, but a vital workaround for a platform that is crucial in the wider development community. I love the Determinate Nix Installer! My interactions with the maintainer have been very positive, too. Can't recommend it enough. :)

grhmc 3y ago
Thank you a lot, pxc! I'm really glad you like it.
flexiondotorg 3y ago
Thanks for the kind words, I'll pass them along to Ana

amelius 3y ago

The fact that you need a third-party installer for Nix or otherwise it will break doesn't inspire much confidence in the whole setup.

emptysongglass 3y ago
It's macOS at fault here.
grhmc 3y ago
We wrote the Determinate Nix Installer with hopes that the greater project will adopt it[1]. Nix itself is fantastic. Improving the upstream Nix installer has been described as "unreasonably expensive and pointless"[2]. Hopefully it is adopted soon! 1. https://determinate.systems/posts/determinate-nix-installer https://determinate.systems/posts/determinate-nix-installer 2. https://discourse.nixos.org/t/nix-survival-mode-macos-upgrades-wont-break-nix-anymore-when-you-install-nix-using-the-determinate-nix-installer/34593/2?u=grahamc https://discourse.nixos.org/t/nix-survival-mode-macos-upgrad...
whateveracct 3y ago
Nix is a very bazaar language culture.
- pxc 3y ago
  This! At any given time, a lot of Nix goodies are being prototyped outside of the Nix and Nixpkgs codebases. So it is with even the installer for Nix itself. But this is really mainly a cultural thing.
deleted 3y ago
[deleted]

oldbbsnickname 3y ago

I'd switch to nix package management on macOS if it were completely isolated to a single user, didn't demand use of sudo, creating /nix or other global changes, or running anything as root. Unfortunately, like Homebrew (except with some changes), MacPorts, and Fink, they seem to be set on global system changes that don't scale.

grhmc 3y ago
Fair enough! Unfortunately that isn't likely to be true any time soon, since macOS doesn't offer the namespacing primitives necessary for that to be possible like it is on Linux. You can bet once it is feasibly a good experience, DetSys will be hot on its tracks to make it happen. Until then, then :).
- oldbbsnickname 3y ago
  Baloney. chroot and execvpe work with mere temporary directories. This is roughly how homebrew does "isolated" builds. You don't need virtualization, but you may want it.
  - grhmc 3y ago
    We don't need virtualization (that'd be easy.) We'd need mount namespacing and bind mounts to place a directory in your home directory at /nix during the process execution. EDIT: User and process namespacing would help a lot as well, to improve build time isolation.
    - oldbbsnickname 3y ago
      You're providing a synthetic filesystem to every process? If so, that sounds absurd and over-engineered.
      - grhmc 3y ago
        No, however because of the general requirement to use /nix/store as a prefix we do support using this method for avoiding global installations.

russelg 3y ago

What do I need to do to switch from an official Nix install to using nix-installer on macOS? Just uninstall the current nix and install using nix-installer? My config is loosely based off https://github.com/dustinlyons/nixos-config https://github.com/dustinlyons/nixos-config if this matters. Using nix-darwin.

zaps 3y ago

Nix seems great for terminal-based development environments where starting a nix-shell sets up all the proper symlinks, env vars, etc. But how does this translate to IDEs like VSCode or IntelliJ? Like if you clone a repo with a shell.nix file, how does the IDE know to use that environment when you, say, use the File > Debug command, instead of executing whatever toolchain that happens to be on your path? Are there plugins available that enable this? I realize you could just open a terminal within the IDE, start a nix shell, and run the equivalent run/debug/etc commands, but I am curious if anything exists to make the experience seamless to the user.

1una 3y ago
> Are there plugins available that enable this? Try https://github.com/direnv/direnv-vscode https://github.com/direnv/direnv-vscode or https://github.com/fehnomenal/intellij-direnv https://github.com/fehnomenal/intellij-direnv
nextos 3y ago
Yes, most Nix users employ https://direnv.net https://direnv.net or the equivalent for your IDE of choice. Emacs for instance has https://github.com/purcell/envrc https://github.com/purcell/envrc which set per-buffer variables. More background in https://github.com/direnv/direnv/wiki/Nix https://github.com/direnv/direnv/wiki/Nix.
- rgoulter 3y ago
  That said, I believe VSCode (and some IDEs) don't necessarily support direnv in all cases.. e.g. you'd want direnv to load before other plugins (and update the PATH), but might not get that.
  - angio 3y ago
    If you start vscode with `code .` after entering the nix shell it will work in most cases.
tripdout 3y ago

yubiox 3y ago

Can you use regular cpanm to install perl modules in nix perl?

eviks 3y ago

Have read many stories re. how complicated the official Nix configuration language is, so was wondering whether there is a plan for an easier config language (if not official, maybe a language that could be transpiled to the official one) similar to how this Rust-based installer is a big progress vs. the clunky bash?

chpatrick 3y ago
It's really not that complicated if you know functional languages. If anything it's quite minimal. It just looks a bit weird because it uses semicolons unlike most functional languages. I kind of wish they just kept JSON/Python syntax for attribute sets and lists. What's complicated (and often not or only lightly documented) is nixpkgs, the package library.
- 0x6c6f6c 3y ago
  To that effect, https://mynixos.com https://mynixos.com has been a really awesome resource for docs. But yeah, the Nix language is definitely not the complicated part. You can get the language down in a day or two with their reference doc. What is difficult is understanding the patterns to NixOS and nixpkg's. Overlays, derivations, packages, flakes.. that's where things get the most confusing. It's a huge step away from other packaging mechanisms like PKGBUILDs but much more powerful and type-safe. But some things just aren't thought of, like home-managers `home.sessionPath` for updating the PATH is _append-only_.. so it literally cannot be used to prepend a preferred bin path. So you follow the typical convention as you might already and include it in the shells rc with `export PATH=/custom/path/bin:$PATH`. Which is the exact kind of safety you'd have today, so it's not worse, it's just another one of the shortcoming of the modules, as opposed to the Nic language itself. The same shortcoming would exist if you used HCL to declaratively define additional PATH paths. It's typically the module design, not the language.