6 ms·

I installed nix on my Mac but quickly backed out due to the complexity. I assumed the nix store would just be an ordinary directory with a tool for managing it,

I installed nix on my Mac but quickly backed out due to the complexity. I assumed the nix store would just be an ordinary directory with a tool for managing it, similar to brew. I discovered it creates a new Unix group, adds a separate APFS volume, installs a daemon. This was too invasive for a tool I was unsure if I even wanted to use, so I uninstalled it. What is the reason for all this machinery? I went with the recommended multi-user install, should I have just used the single user mode instead?

nodejsthrowaway 4y ago
edolstra 4y ago
It's because Nix wants to install into /nix. Once upon a time doing "sudo mkdir /nix" wasn't a problem, but recent macOS releases have made that very hard. Nix could switch to an alternate location on macOS (e.g. /opt/nix) but that has a lot of downsides for interoperability with other systems.
- qudat 4y ago
  Yikes, I had no idea. What a tough pill to swallow all because the directory nix runs out of is hardcoded.
  - kaba0 4y ago
    It’s not actually hardcoded, but you would have to compile everything from scratch as the hash would depend on that as well, throwing out the whole binary cache.
    - wmertens 4y ago
      Actually, there's quite a few hard-coded references to /nix in nixpkgs I believe. It's possible but a little more work than just changing the prefix.
      - grhmc 4y ago
        There may be, but at least one BigCorp deploys Nix to a different prefix.
  - vamega 4y ago
    It's not hardcoded, but you end can't take advantage of the binary caches if you change the directory. A company could certainly create their own binary cache and distribute that to it's users.
    - rnhmjoj 4y ago
      Not entirely true, there are many ways in which you can use a custom location and still take advantage of the binary cache. You can do it with chroot, file system namespaces, bind mounts and so on. There's also a nice user friendly tool that does exactly this [1]. [1]: https://github.com/DavHau/nix-portable https://github.com/DavHau/nix-portable
  - grhmc 4y ago
    On Linux, users can put their Nix store in their home directory or other places and at run-time Nix remaps the directory using user namespaces. Unfortunately this isn't workable on macOS: the kernel doesn't support the features we need. Using /nix and a separate group and daemon means the store can be read-only and be protected from modification in several ways. This is pretty helpful, as a lot of tools try very hard to write "next to" where they are installed -- corrupting the Nix store. I sort of wonder if it would be more palatable if the Nix installer was a bit less in your face about what's going on? This would be similar to how Docker's works.
    - pseudalopex 4y ago
      It would be less palatable when I found out. The group is fine. Why the daemon when other package managers use sudo is unclear. Even Homebrew moved to /opt.
      - lilyball 4y ago
        Other package managers are okay with requiring sudo because they install stuff globally. Nix doesn't have that restriction, you can use it for local stuff, temporary shell environments, etc. So you need non-admin users to be able to use it too, and even admin users need to be able to use it without using sudo. For example, when entering a nix shell, you don't want the shell to run as root. Or when using direnv. Or just when using it as part of your build system. Nix is a package manager, yes, but it's more than that, it's a generalized build system.
        pseudalopex 4y ago
        Nix isn't the only package manager for home directories or source packages. Working in a home directory doesn't require sudo or a daemon. sudo doesn't mean run everything as root.
        soraminazuki 4y ago
        You missed the crucial point: > So you need non-admin users to be able to use it too The build daemon and the user are used for privilege separation. The separation goes both ways. Users can't write directly to /nix/store and Nix can't write outside of /nix/store during build. If anything, it's there to make things less invasive. It's nothing like the Docker daemon, which is a proxy for root. Additionally, the daemon doesn't do anything unless users request that a package be built.
  - ParetoOptimal 4y ago
    > What a tough pill to swallow all because the directory nix runs out of is hardcoded. Yikes? Well then... how would you solve remote binary caching with something like Nix on a platform such as OSX without userns remapping support?
  - Macha 4y ago
    This is inherent to having binary packages. Binaries compiled to look for libraries in one location cannot just be copied to a system with a different layout. Nix also can't just use the OS conventions, since part of the point of Nix is that it does not use the global system state and only its own isolated, reproducible world. So then they had to make a decision and decided to just use /nix everywhere which worked fine for some years. And now MacOS has changed, choosing something else would invalidate all historical binary packages.
rgoulter 4y ago
> What is the reason for all this machinery? Nix wants to use /nix, but Apple wants macOS to be locked down and secure. The compromise was putting /nix on a separate volume. > I discovered it creates a new Unix group, ... installs a daemon This is at about the same level as Docker, fwiw.
- chrisseaton 4y ago
  > The compromise was putting /nix on a separate directory. Why does a separate directory need a whole other volume? These are just files, right? Why do they need to be on a separate volume?
  - ParetoOptimal 4y ago
    OSX requires mounting root folders like /nix with synthetic.conf IIRC.
    - chrisseaton 4y ago
      Right but the comment said they compromised on using a different directory in the end, so it could be under /opt like Homebrew, and not a root directory.
      - rgoulter 4y ago
        One of Nix's benefits is it aims to isolate where programs are installed; so you can have multiple versions of the same program available without conflict. (e.g. programs built with different feature flags, or multiple minor versions). Nix achieves this by storing packages in a path as some hash of its inputs. -- This then allows either compiling or downloading a package, with confidence that it will behave the same way regardless. But, since the files were under /nix, if you put them under /opt, then you wouldn't be able to make use of the compilation caches for /nix.
        chrisseaton 4y ago
0x457 4y ago
APFS volume is required to have a read-only nix-store at /nix. The daemon is only required in multi-user installation, and you can opt out of it (by using a single-user install). New groups are for letting people submit jobs to builders in multi-user mode.
- Macha 4y ago
  Note that the multi-user installation is these days the only suggested approach by the Nix team for new MacOS installations - the old instructions for single user installation on MacOS have been removed from their site.
scoates 4y ago
It's because builds are completely reproducible, and therefore they have to be built in the same place (in `/nix`), and you can't create directories off of `/` on modern Mac setups.
ParetoOptimal 4y ago
The install process was simpler before recent OSX updates. > I discovered it creates a new Unix group, adds a separate APFS volume, installs a daemon. This was too invasive for a tool I was unsure if I even wanted to use, so I uninstalled it. Since you'd already installed it, wouldn't trying it in some capacity before uninstalling it have made sense? > What is the reason for all this machinery Enforcing reproducibility basically. I don't know all but... The daemon and APFS volume otherwise readonly /nix that only the daemon can write to can't be created. The group is probably for the daemon to be able to write to /nix. The path /nix is important because the remote binary cache paths will miss otherwise and you'll compile everything from source. > I went with the recommended multi-user install, should I have just used the single user mode instead? I'm guessing it would work in the way you want, but I always opt for the daemon.
- ridiculous_fish 4y ago
  > Since you'd already installed it, wouldn't trying it in some capacity before uninstalling it have made sense? Fair observation. I installed nix as a prerequisite for DevBox, discussed here https://news.ycombinator.com/item?id=32600821 https://news.ycombinator.com/item?id=32600821 . I thought DevBox sounded really cool (and still do!), but the Quickstart took frustratingly long, and ended up not working. Faced with the prospect of debugging it, I opted to cut my losses and uninstall it instead. That said I'm very much open to trying nix again in the future. Also I want to acknowledge how much effort went into getting /nix to work on the Mac; it appears that was a heavy lift indeed.
grhmc 4y ago
Single-user Nix installs are lighter weight, but also have a major compromise: the store is writable by the user. This means it is pretty easy for software you're using to accidentally change the store underneath Nix, and create unpredictable and unreproducible behavior. I think the multi-user install is a better Nix experience, even if the install process is spookier. Like is mentioned elsewhere, Docker does similar contortions to install itself. I wonder if it would have been more palatable if the Nix installer was less forward about what it is doing?
- ParetoOptimal 4y ago
  > I wonder if it would have been more palatable if the Nix installer was less forward about what it is doing? I bet most wouldn't consider it spooky if the installer didn't print it out. If docker started printing it, I bet lots of people would complain it seems similarly complex.
  - rgoulter 4y ago
    I recall the sqlite changed their tmpfiles on Windows from "sqlite" to "etilqs".
tadfisher 4y ago
> What is the reason for all this machinery? > new Unix group This is used for the daemon, so it doesn't run as root and expose your system to Nix build code. > adds a separate APFS volume I think this is required because of macOS security restrictions preventing direct modification of the root directory. The Nix store has to be housed in /nix because all references to runtime dependencies in the store are absolute paths in /nix, and that can't really change per system because it would break caching and reproducibility. The separate volume is added to /etc/synthetic.conf so it can live in /nix. > installs a daemon For multi-user installs, this allows non-privileged users to add build outputs to the Nix store, which ultimately allows these users to share build outputs. More info here: https://nixos.org/manual/nix/stable/installation/multi-user.html https://nixos.org/manual/nix/stable/installation/multi-user....
- microtonal 4y ago
  The Nix store has to be housed in /nix because all references to runtime dependencies in the store are absolute paths in /nix, This is true, but in some sense Nix on Apple Silicon was a missed opportunity. It started as a blank slate (fresh binary cache) and it would've been a good opportunity to move the store to a writable path like /opt/nix. This would have solved the whole dance needed with volumes and synthetic.conf. I know that there are infrastructure issues (like Hydra using a single Nix store), but it would've made the macOS Nix story so much better. It's a shame that only Apple can make firmlinks, because that would've been another possible solution (/nix could be a firmlink to the actual store location). The same problem occurs on e.g. Fedora SilverBlue, because you can also not make arbitrary root directories. But at least on Linux, you don't want to throw away more than one decade of a x86_64 binary cache.
  - wmertens 4y ago
    Actually the cache isn't that important, most binaries get rebuilt quite often due to their dependencies changing. Although it is my hope that rfc 17 eventually makes it through: https://github.com/wmertens/rfcs/blob/master/rfcs/0017-intensional-store.md https://github.com/wmertens/rfcs/blob/master/rfcs/0017-inten...
joshka 4y ago
The meta answer to the question is this info belongs at the non-existent doc - https://nixos.org/manual/nix/stable/installation/macos.html https://nixos.org/manual/nix/stable/installation/macos.html, but no-one has put it there yet.

nodejsthrowaway 4y ago

edolstra 4y ago

It's because Nix wants to install into /nix. Once upon a time doing "sudo mkdir /nix" wasn't a problem, but recent macOS releases have made that very hard. Nix could switch to an alternate location on macOS (e.g. /opt/nix) but that has a lot of downsides for interoperability with other systems.

qudat 4y ago
Yikes, I had no idea. What a tough pill to swallow all because the directory nix runs out of is hardcoded.
- kaba0 4y ago
  It’s not actually hardcoded, but you would have to compile everything from scratch as the hash would depend on that as well, throwing out the whole binary cache.
  - wmertens 4y ago
    Actually, there's quite a few hard-coded references to /nix in nixpkgs I believe. It's possible but a little more work than just changing the prefix.
    - grhmc 4y ago
      There may be, but at least one BigCorp deploys Nix to a different prefix.
- vamega 4y ago
  It's not hardcoded, but you end can't take advantage of the binary caches if you change the directory. A company could certainly create their own binary cache and distribute that to it's users.
  - rnhmjoj 4y ago
    Not entirely true, there are many ways in which you can use a custom location and still take advantage of the binary cache. You can do it with chroot, file system namespaces, bind mounts and so on. There's also a nice user friendly tool that does exactly this [1]. [1]: https://github.com/DavHau/nix-portable https://github.com/DavHau/nix-portable
- grhmc 4y ago
  On Linux, users can put their Nix store in their home directory or other places and at run-time Nix remaps the directory using user namespaces. Unfortunately this isn't workable on macOS: the kernel doesn't support the features we need. Using /nix and a separate group and daemon means the store can be read-only and be protected from modification in several ways. This is pretty helpful, as a lot of tools try very hard to write "next to" where they are installed -- corrupting the Nix store. I sort of wonder if it would be more palatable if the Nix installer was a bit less in your face about what's going on? This would be similar to how Docker's works.
  - pseudalopex 4y ago
    It would be less palatable when I found out. The group is fine. Why the daemon when other package managers use sudo is unclear. Even Homebrew moved to /opt.
    - lilyball 4y ago
      Other package managers are okay with requiring sudo because they install stuff globally. Nix doesn't have that restriction, you can use it for local stuff, temporary shell environments, etc. So you need non-admin users to be able to use it too, and even admin users need to be able to use it without using sudo. For example, when entering a nix shell, you don't want the shell to run as root. Or when using direnv. Or just when using it as part of your build system. Nix is a package manager, yes, but it's more than that, it's a generalized build system.
      - pseudalopex 4y ago
        Nix isn't the only package manager for home directories or source packages. Working in a home directory doesn't require sudo or a daemon. sudo doesn't mean run everything as root.
        soraminazuki 4y ago
        You missed the crucial point: > So you need non-admin users to be able to use it too The build daemon and the user are used for privilege separation. The separation goes both ways. Users can't write directly to /nix/store and Nix can't write outside of /nix/store during build. If anything, it's there to make things less invasive. It's nothing like the Docker daemon, which is a proxy for root. Additionally, the daemon doesn't do anything unless users request that a package be built.
- ParetoOptimal 4y ago
  > What a tough pill to swallow all because the directory nix runs out of is hardcoded. Yikes? Well then... how would you solve remote binary caching with something like Nix on a platform such as OSX without userns remapping support?
- Macha 4y ago
  This is inherent to having binary packages. Binaries compiled to look for libraries in one location cannot just be copied to a system with a different layout. Nix also can't just use the OS conventions, since part of the point of Nix is that it does not use the global system state and only its own isolated, reproducible world. So then they had to make a decision and decided to just use /nix everywhere which worked fine for some years. And now MacOS has changed, choosing something else would invalidate all historical binary packages.

rgoulter 4y ago

> What is the reason for all this machinery? Nix wants to use /nix, but Apple wants macOS to be locked down and secure. The compromise was putting /nix on a separate volume. > I discovered it creates a new Unix group, ... installs a daemon This is at about the same level as Docker, fwiw.

chrisseaton 4y ago
> The compromise was putting /nix on a separate directory. Why does a separate directory need a whole other volume? These are just files, right? Why do they need to be on a separate volume?
- ParetoOptimal 4y ago
  OSX requires mounting root folders like /nix with synthetic.conf IIRC.
  - chrisseaton 4y ago
    Right but the comment said they compromised on using a different directory in the end, so it could be under /opt like Homebrew, and not a root directory.
    - rgoulter 4y ago
      One of Nix's benefits is it aims to isolate where programs are installed; so you can have multiple versions of the same program available without conflict. (e.g. programs built with different feature flags, or multiple minor versions). Nix achieves this by storing packages in a path as some hash of its inputs. -- This then allows either compiling or downloading a package, with confidence that it will behave the same way regardless. But, since the files were under /nix, if you put them under /opt, then you wouldn't be able to make use of the compilation caches for /nix.
      - chrisseaton 4y ago

0x457 4y ago

APFS volume is required to have a read-only nix-store at /nix. The daemon is only required in multi-user installation, and you can opt out of it (by using a single-user install). New groups are for letting people submit jobs to builders in multi-user mode.

Macha 4y ago
Note that the multi-user installation is these days the only suggested approach by the Nix team for new MacOS installations - the old instructions for single user installation on MacOS have been removed from their site.

scoates 4y ago

It's because builds are completely reproducible, and therefore they have to be built in the same place (in `/nix`), and you can't create directories off of `/` on modern Mac setups.

ParetoOptimal 4y ago

The install process was simpler before recent OSX updates. > I discovered it creates a new Unix group, adds a separate APFS volume, installs a daemon. This was too invasive for a tool I was unsure if I even wanted to use, so I uninstalled it. Since you'd already installed it, wouldn't trying it in some capacity before uninstalling it have made sense? > What is the reason for all this machinery Enforcing reproducibility basically. I don't know all but... The daemon and APFS volume otherwise readonly /nix that only the daemon can write to can't be created. The group is probably for the daemon to be able to write to /nix. The path /nix is important because the remote binary cache paths will miss otherwise and you'll compile everything from source. > I went with the recommended multi-user install, should I have just used the single user mode instead? I'm guessing it would work in the way you want, but I always opt for the daemon.

ridiculous_fish 4y ago
> Since you'd already installed it, wouldn't trying it in some capacity before uninstalling it have made sense? Fair observation. I installed nix as a prerequisite for DevBox, discussed here https://news.ycombinator.com/item?id=32600821 https://news.ycombinator.com/item?id=32600821 . I thought DevBox sounded really cool (and still do!), but the Quickstart took frustratingly long, and ended up not working. Faced with the prospect of debugging it, I opted to cut my losses and uninstall it instead. That said I'm very much open to trying nix again in the future. Also I want to acknowledge how much effort went into getting /nix to work on the Mac; it appears that was a heavy lift indeed.

grhmc 4y ago

Single-user Nix installs are lighter weight, but also have a major compromise: the store is writable by the user. This means it is pretty easy for software you're using to accidentally change the store underneath Nix, and create unpredictable and unreproducible behavior. I think the multi-user install is a better Nix experience, even if the install process is spookier. Like is mentioned elsewhere, Docker does similar contortions to install itself. I wonder if it would have been more palatable if the Nix installer was less forward about what it is doing?

ParetoOptimal 4y ago
> I wonder if it would have been more palatable if the Nix installer was less forward about what it is doing? I bet most wouldn't consider it spooky if the installer didn't print it out. If docker started printing it, I bet lots of people would complain it seems similarly complex.
- rgoulter 4y ago
  I recall the sqlite changed their tmpfiles on Windows from "sqlite" to "etilqs".

tadfisher 4y ago

> What is the reason for all this machinery? > new Unix group This is used for the daemon, so it doesn't run as root and expose your system to Nix build code. > adds a separate APFS volume I think this is required because of macOS security restrictions preventing direct modification of the root directory. The Nix store has to be housed in /nix because all references to runtime dependencies in the store are absolute paths in /nix, and that can't really change per system because it would break caching and reproducibility. The separate volume is added to /etc/synthetic.conf so it can live in /nix. > installs a daemon For multi-user installs, this allows non-privileged users to add build outputs to the Nix store, which ultimately allows these users to share build outputs. More info here: https://nixos.org/manual/nix/stable/installation/multi-user.html https://nixos.org/manual/nix/stable/installation/multi-user....

microtonal 4y ago
The Nix store has to be housed in /nix because all references to runtime dependencies in the store are absolute paths in /nix, This is true, but in some sense Nix on Apple Silicon was a missed opportunity. It started as a blank slate (fresh binary cache) and it would've been a good opportunity to move the store to a writable path like /opt/nix. This would have solved the whole dance needed with volumes and synthetic.conf. I know that there are infrastructure issues (like Hydra using a single Nix store), but it would've made the macOS Nix story so much better. It's a shame that only Apple can make firmlinks, because that would've been another possible solution (/nix could be a firmlink to the actual store location). The same problem occurs on e.g. Fedora SilverBlue, because you can also not make arbitrary root directories. But at least on Linux, you don't want to throw away more than one decade of a x86_64 binary cache.
- wmertens 4y ago
  Actually the cache isn't that important, most binaries get rebuilt quite often due to their dependencies changing. Although it is my hope that rfc 17 eventually makes it through: https://github.com/wmertens/rfcs/blob/master/rfcs/0017-intensional-store.md https://github.com/wmertens/rfcs/blob/master/rfcs/0017-inten...

joshka 4y ago

The meta answer to the question is this info belongs at the non-existent doc - https://nixos.org/manual/nix/stable/installation/macos.html https://nixos.org/manual/nix/stable/installation/macos.html, but no-one has put it there yet.