5 ms·

Unfortunately there's a pretty annoying bug with MacOS which resulted from Apple making /nix non-writable by default. And since /nix is hard coded in all the ca

Unfortunately there's a pretty annoying bug with MacOS which resulted from Apple making /nix non-writable by default. And since /nix is hard coded in all the cached packages it's not easy to fix. This is one big thing that's preventing us from adopting nix https://github.com/NixOS/nix/issues/2925 https://github.com/NixOS/nix/issues/2925

geofft 6y ago
I'm trying to understand why macOS can't use a different path (since it's a different OS anyway, and you can't run Linux Nix binaries on macOS). From the end of the thread, it sounds like > The main consequence of using a separate prefix for macOS is that you can't have Hydra jobsets anymore containing jobs for macOS and Linux. It would also make it harder to deploy from macOS to Linux. i.e., if the same package builds for both Linux and macOS, you can't write a single shell script that runs on both Linux Nix and macOS Nix and calls that package? Maybe I'm misunderstanding how people deploy things, but is that a huge problem in practice? (I don't quite follow the thing about Hydra jobsets. Aren't you building once for Linux and once for macOS anyway? Figuring out how to make the build do that seems like it would subsume any complexity of using /nix vs. /opt/nix.)
- takeda 6y ago
  From looking at PR seems that it will utilize some extra space (for derivations that aren't really OS specific) and there might be problems wih deploying from os x to linux, but feels like they are planning to go that route ultimately.
- derefr 6y ago
  Perhaps they hardcode full paths into whatever Nix calls its formula files (which are then, if I understand correctly, referred to by their cryptographic hashes, meaning that a file with a different path would have a different hash, and so break the whole hierarchy of formula-to-formula dependency references above it.)
- elbear 6y ago
  You can use a different path, but then you don't have access to the binary cache, because of the mismatch between your path and the path found in the cached binaries. That's how I understand it.
aeontech 6y ago
It's not a bug in macOS, it's a security feature. However, you can use synthetic firmlinks to define arbitrary root-level paths that _are_ writable. See `man synthetic.conf` and check out https://derflounder.wordpress.com/2020/01/18/creating-root-level-directories-and-symbolic-links-on-macos-catalina/ https://derflounder.wordpress.com/2020/01/18/creating-root-l... for an example usage. PS: It looks like nix has a PR open to test that solution here: https://github.com/NixOS/nix/pull/3212 https://github.com/NixOS/nix/pull/3212
- danieldk 6y ago
  However, you can use synthetic firmlinks to define arbitrary root-level paths that _are_ writable. See `man synthetic.conf` No, synthetic.conf only creates directories or symlinks. If it actually created firmlinks, the problem would probably be much smaller. The problem with symlinks is that some builds use realpath. If /nix was a symlink to e.g. /usr/local/nix, this would result in /usr/local/nix being used in this builds. This leads to various build problems [1] and can result in the wrong paths ending up in binaries. If this happens on a builder, the resulting binaries are not portable between macOS + Nix machines. So, instead the only way to get it currently working is to use synthetic.conf to create a new top-level directory and mount a APFS volume with the Nix store there. If Apple actually allowed creating firmlinks with synthetic.conf, the Nix installation would be drastically simplified. /nix could just be a firmlink to e.g. /usr/local/nix, since realpath does not report the actual path on firmlinks. [1] https://github.com/NixOS/nix/issues/2926#issuecomment-559989390 https://github.com/NixOS/nix/issues/2926#issuecomment-559989...
  - aeontech 6y ago
    I stand corrected, I must have misunderstood the doc.
endgame 6y ago
Buried in that issue is a workaround, although GitHub's UI hides it from you: https://github.com/NixOS/nix/issues/2925#issuecomment-604501661 https://github.com/NixOS/nix/issues/2925#issuecomment-604501...
- MaxGabriel 6y ago
  Several people in my company use the separate volume workaround and it's been totally painless. Nix has sharp edges but this one is less of a concern than you'd think ime
tome 6y ago
Hardcoding /nix is a very big downside of Nix for me. I understand why this makes other things easier, but it's a huge compromise on flexibility.

geofft 6y ago

I'm trying to understand why macOS can't use a different path (since it's a different OS anyway, and you can't run Linux Nix binaries on macOS). From the end of the thread, it sounds like > The main consequence of using a separate prefix for macOS is that you can't have Hydra jobsets anymore containing jobs for macOS and Linux. It would also make it harder to deploy from macOS to Linux. i.e., if the same package builds for both Linux and macOS, you can't write a single shell script that runs on both Linux Nix and macOS Nix and calls that package? Maybe I'm misunderstanding how people deploy things, but is that a huge problem in practice? (I don't quite follow the thing about Hydra jobsets. Aren't you building once for Linux and once for macOS anyway? Figuring out how to make the build do that seems like it would subsume any complexity of using /nix vs. /opt/nix.)

takeda 6y ago
From looking at PR seems that it will utilize some extra space (for derivations that aren't really OS specific) and there might be problems wih deploying from os x to linux, but feels like they are planning to go that route ultimately.
derefr 6y ago
Perhaps they hardcode full paths into whatever Nix calls its formula files (which are then, if I understand correctly, referred to by their cryptographic hashes, meaning that a file with a different path would have a different hash, and so break the whole hierarchy of formula-to-formula dependency references above it.)
elbear 6y ago
You can use a different path, but then you don't have access to the binary cache, because of the mismatch between your path and the path found in the cached binaries. That's how I understand it.

aeontech 6y ago

It's not a bug in macOS, it's a security feature. However, you can use synthetic firmlinks to define arbitrary root-level paths that _are_ writable. See `man synthetic.conf` and check out https://derflounder.wordpress.com/2020/01/18/creating-root-level-directories-and-symbolic-links-on-macos-catalina/ https://derflounder.wordpress.com/2020/01/18/creating-root-l... for an example usage. PS: It looks like nix has a PR open to test that solution here: https://github.com/NixOS/nix/pull/3212 https://github.com/NixOS/nix/pull/3212

danieldk 6y ago
However, you can use synthetic firmlinks to define arbitrary root-level paths that _are_ writable. See `man synthetic.conf` No, synthetic.conf only creates directories or symlinks. If it actually created firmlinks, the problem would probably be much smaller. The problem with symlinks is that some builds use realpath. If /nix was a symlink to e.g. /usr/local/nix, this would result in /usr/local/nix being used in this builds. This leads to various build problems [1] and can result in the wrong paths ending up in binaries. If this happens on a builder, the resulting binaries are not portable between macOS + Nix machines. So, instead the only way to get it currently working is to use synthetic.conf to create a new top-level directory and mount a APFS volume with the Nix store there. If Apple actually allowed creating firmlinks with synthetic.conf, the Nix installation would be drastically simplified. /nix could just be a firmlink to e.g. /usr/local/nix, since realpath does not report the actual path on firmlinks. [1] https://github.com/NixOS/nix/issues/2926#issuecomment-559989390 https://github.com/NixOS/nix/issues/2926#issuecomment-559989...
- aeontech 6y ago
  I stand corrected, I must have misunderstood the doc.

endgame 6y ago

Buried in that issue is a workaround, although GitHub's UI hides it from you: https://github.com/NixOS/nix/issues/2925#issuecomment-604501661 https://github.com/NixOS/nix/issues/2925#issuecomment-604501...

MaxGabriel 6y ago
Several people in my company use the separate volume workaround and it's been totally painless. Nix has sharp edges but this one is less of a concern than you'd think ime

tome 6y ago

Hardcoding /nix is a very big downside of Nix for me. I understand why this makes other things easier, but it's a huge compromise on flexibility.