8 ms·

> eliminating the filesystem (other than home and var) as a big ol' chunk of globally mutable state is so liberating I'm about a week into the (very painful) p

> eliminating the filesystem (other than home and var) as a big ol' chunk of globally mutable state is so liberating I'm about a week into the (very painful) process of switching to NixOS. This is pretty much the promise of NixOS that got me interested, but it seems to be that it's not really true. NixOS is just running a regular kernel that does regular linux security things. If you want AppArmor or SELinux you still have to configure it yourself. If you want a sandbox on NixOS, your options are still bubblewrap/firejail, proot, or flatpak. Or of course full virtualization with libvirt. The NixOS "containers" are just systemd-nspawn which (if I understand correctly) doesn't really offer more security than a properly permissioned user. I suspect that if you installed a malicious binary in a NixOS package, you'd be just as compromised as you would installing something malicious from AUR.

mitchellh 3y ago
The Nix store `/nix` is readonly except by the Nix build users. So, if you’re using Nix derivations for everything (the end goal), then rogue processes cannot interfere in any way with files outside of the chroot-like environment the build process creates. The writable directories (your home dir and var, as the parent stated) are still “vulnerable” and a program can run anything they want of course (bound by typical Linux/Unix rules). Nix isn’t a containerization/sandboxing technology, but it does remove any fear of installing software overwriting files you wanted, including OS level (and kernel) upgrades.
- throwing_away 3y ago
  > but it does remove any fear of installing software overwriting files you wanted, including OS level (and kernel) upgrades. I understand that's "true" in a theoretical way because the store is read-only and it is all hashed. But the hashes aren't routinely checked by some kind of hypervisor, and root can still overwrite things in the store. The "fear of installing software overwriting files you wanted" essentially comes down to config file management (unhappy accidents) and malware. You should have config file management in git already, so I don't feel like NixOS needs to solve that. I was hoping it would solve the problem of random software being able to obtain root and not ransomware me, but it practically doesn't solve that any better than any other distro. I want to be missing something. I've invested a lot of time learning about Nix for the last week and my system is finally working, but I just got to the sandboxing/security portion of my install and the threat model seems broken.
  - xyzzy_plugh 3y ago
    It seems like you misinterpreted the isolation that's advertised. There are security benefit but the isolation provided is predominantly about deterministic and reproducible runtime environments. There's nothing particularly novel happening at the OS level compared to, say, Debian, but the difference is in how you arrived at the current state. You're free to sprinkle whatever other security bits you are fond of.
    - throwing_away 3y ago
      > deterministic and reproducible runtime environments But is there a point to having what you believe are deterministic and reproducible runtimes if the environment used to build them doesn't protect against malware in a build from escaping into the build system?
      - zaphar 3y ago
        There are multiple non security reasons to want deterministic and reproducible runtime environments. 1. Everything you need to describe and build a project is obtained by checking it out. 2. No more, works on my machine but not yours problems. 3. No more weird library dependency conflicts. I've run into all of these and I find them all frustrating when it happens.
        throwing_away 3y ago
        These are good things and possibly make the struggle of NixOS worth it. But the impression the community gives is very much that you can always rollback and everything is in its own sandbox, which is sort of true, but not at all true as soon as malware happens. I've never really had a huge problem rolling back an ubuntu or arch update when something breaks, so I'm surprised at the amount of effort people are expending for just this feature with no additional security.
        pxc 3y ago
        > But the impression the community gives is very much that you can always rollback and everything is in its own sandbox, which is sort of true, but not at all true as soon as malware happens. I can see why if you do malware research or something like that, you might hear about the rollbacks capability and get your hopes up in a certain way, but that's not really the use case. NixOS is nice for running untrusted/suspect software in a few ways I can think of, though. (They don't exactly make NixOS a security tool, but I think maybe you could leverage them to build one.) 1. If a NixOS system is compromised, blowing it away and installing from clean media is cheap compared to on other operating systems, since you can retain more of your special configuration. Reinstalling is a bit more like a snapshot restore, at least in terms of the systemwide setup (users, running services and their configurations, and installed packages). 2. NixOS does make it really easy to spin up a clone of your running configuration in a VM instead of directly switching to that config on the local system. 3. The Nix sandbox is a pretty nice place to perform builds from big repos where packages can run arbitrary hooks at build time, like PyPI and NPM, since you can have it build in chroots with temporary build users, no network access, and maybe some other nice things. There is also actually at least one Nix-based OS trying to achieve new things in security research out there, Spectrum: https://spectrum-os.org/ https://spectrum-os.org/ > I'm surprised at the amount of effort people are expending for just this feature with no additional security. NixOS (and Nix more generally) doesn't have a singular feature like that driving its usage or development forward, even though rollbacks is a really nice one that will often come to mind when you ask NixOS users what they like about running NixOS. But if I had to name contenders for the top one or two 'biggest differentiators' from other tools/paradigms that let you achieve similar effects, like other configuration management systems or copy-on-write filesystem snapshotting, I'd say it's the totalizing way that NixOS integrates those features. Because NixOS generates the configuration of the whole system, it gets to avoid having to inspect most of the system's state, and generally handles the bits of transitioning between configurations that do require inspecting and reasoning about the state of the system quickly and pretty well. There's just a smaller surface area there. Similarly, you're just more likely to be able to easily roll back with NixOS because those features are built into all of the normal system/package management operations, and leveraging those things is generally the path of least resistance to changing the system. You end up being able to count on them more 'by default'— you're much less likely to make an important change and have a gap without a snapshot. The garbage collection system makes clearing the unused ones data easier (imo). The general reproducibility also gives you multiple sort of layers of intervention for rolling back— even if you do collect all your past generations, your version control system becomes another reliable way of 'rolling back'. Both of those ways of going back and forth through iterations of your configuration can be further combined with similar interventions at other layers, like dotfile management via Home Manager, snapshotting filesystems for unmanaged files, selective persistence via something like Impermanence, etc. These things can add up a system where the kind of ad-hoc changes that might leak through your state management tools (snapper, etckeeper, dotfile management, etc.) become a radical departure from the way you regularly work. Another differentiator here is maybe the generality: when you Nixify, you sometimes have to do a lot of work up front just to get things working on any deployment target, but the marginal work to go from a NixOS setup to some other kind— generating identical container images, preparing a VM for local use, running your config on macOS, partially sharing your desktop configuration with a server, letting a friend or colleague experiment with or debug your exact setup, preparing an AMI, etc.— is lower, and decreases with each further investment you make in the Nix universe. Different aspects of that inevitably end up being valuable, impressive, or delightful to different users. Taken alone, none of them might seem incomparably compelling over alternative approaches.
- fbdab103 3y ago
  >...but it does remove any fear of installing software overwriting files you wanted, including OS level (and kernel) upgrades. I guess that is nice to know my Python installation cannot be deleted by malware, but I am more concerned about a process stealing my .ssh keys.
  - arjvik 3y ago
    Honestly, exactly recreating my byzantine python installation(s) would probably be impossible [0] (In all seriousness, after switching to Arch I've actually solved this problem by simply installing python packages I need from the AUR) [0]: https://xkcd.com/1987/ https://xkcd.com/1987/
cookiengineer 3y ago
I can recommend reading the systemd manual entries (e.g. man systemd.exec). SystemD meanwhile has a lot of options for managing a seccomp based sandbox, e.g. various protect options for the filesystem, mounting critical things as read-only, simulating a chroot with its own fake-root user etc.pp. You can also manage the capabilities of a binary from there, so it's actually integrated down the kernel stack. However, as you mentioned, the lack of an official "profile database" for common packages/software makes it just as useless as the other tools. I wish we had a repo where all the things come together and people can just do something like "autosandbox apache2" and it will do the rest.
- throwing_away 3y ago
  Thanks. I'm learning about this today and I'm beginning to suspect all the extra isolation software is not really useful if you configure AppArmor and SystemD properly per service. The space between "full virtual machine" and "unix permission model" is vast and confusing. I would have thought that because everything is hashed on nix, it would be trivial to spin up full "virtual machines" without consuming mountains of disk space, but that does not seem to be an option.
- progval 3y ago
  Firejail does this. The profile database is the two "profile" directories in https://github.com/netblue30/firejail/tree/master/etc https://github.com/netblue30/firejail/tree/master/etc
- D_Sam 3y ago
  Sorry… I see no other way to contact you. I saw here in one of your previous comments that you were able to put 32gb of memory on a T440p… can you tell how!? If possible please dm me. Thanks.
nextos 3y ago
> If you want a sandbox on NixOS, your options are still [...] Exactly, I have brought this up in the Nix forum a few times, but developers don't seem to see the value. Really frustrating and a bit ironic given that sandboxing should be quite appealing to a crowd that is attracted to Nix. Guix does implement some a variety of sandboxing options.
paulddraper 3y ago
NixOS isn't a security solution. If you expect that, you will be disappointed. It's isolated for good actors.

mitchellh 3y ago

The Nix store `/nix` is readonly except by the Nix build users. So, if you’re using Nix derivations for everything (the end goal), then rogue processes cannot interfere in any way with files outside of the chroot-like environment the build process creates. The writable directories (your home dir and var, as the parent stated) are still “vulnerable” and a program can run anything they want of course (bound by typical Linux/Unix rules). Nix isn’t a containerization/sandboxing technology, but it does remove any fear of installing software overwriting files you wanted, including OS level (and kernel) upgrades.

throwing_away 3y ago
> but it does remove any fear of installing software overwriting files you wanted, including OS level (and kernel) upgrades. I understand that's "true" in a theoretical way because the store is read-only and it is all hashed. But the hashes aren't routinely checked by some kind of hypervisor, and root can still overwrite things in the store. The "fear of installing software overwriting files you wanted" essentially comes down to config file management (unhappy accidents) and malware. You should have config file management in git already, so I don't feel like NixOS needs to solve that. I was hoping it would solve the problem of random software being able to obtain root and not ransomware me, but it practically doesn't solve that any better than any other distro. I want to be missing something. I've invested a lot of time learning about Nix for the last week and my system is finally working, but I just got to the sandboxing/security portion of my install and the threat model seems broken.
- xyzzy_plugh 3y ago
  It seems like you misinterpreted the isolation that's advertised. There are security benefit but the isolation provided is predominantly about deterministic and reproducible runtime environments. There's nothing particularly novel happening at the OS level compared to, say, Debian, but the difference is in how you arrived at the current state. You're free to sprinkle whatever other security bits you are fond of.
  - throwing_away 3y ago
    > deterministic and reproducible runtime environments But is there a point to having what you believe are deterministic and reproducible runtimes if the environment used to build them doesn't protect against malware in a build from escaping into the build system?
    - zaphar 3y ago
      There are multiple non security reasons to want deterministic and reproducible runtime environments. 1. Everything you need to describe and build a project is obtained by checking it out. 2. No more, works on my machine but not yours problems. 3. No more weird library dependency conflicts. I've run into all of these and I find them all frustrating when it happens.
      - throwing_away 3y ago
        These are good things and possibly make the struggle of NixOS worth it. But the impression the community gives is very much that you can always rollback and everything is in its own sandbox, which is sort of true, but not at all true as soon as malware happens. I've never really had a huge problem rolling back an ubuntu or arch update when something breaks, so I'm surprised at the amount of effort people are expending for just this feature with no additional security.
        pxc 3y ago
        > But the impression the community gives is very much that you can always rollback and everything is in its own sandbox, which is sort of true, but not at all true as soon as malware happens. I can see why if you do malware research or something like that, you might hear about the rollbacks capability and get your hopes up in a certain way, but that's not really the use case. NixOS is nice for running untrusted/suspect software in a few ways I can think of, though. (They don't exactly make NixOS a security tool, but I think maybe you could leverage them to build one.) 1. If a NixOS system is compromised, blowing it away and installing from clean media is cheap compared to on other operating systems, since you can retain more of your special configuration. Reinstalling is a bit more like a snapshot restore, at least in terms of the systemwide setup (users, running services and their configurations, and installed packages). 2. NixOS does make it really easy to spin up a clone of your running configuration in a VM instead of directly switching to that config on the local system. 3. The Nix sandbox is a pretty nice place to perform builds from big repos where packages can run arbitrary hooks at build time, like PyPI and NPM, since you can have it build in chroots with temporary build users, no network access, and maybe some other nice things. There is also actually at least one Nix-based OS trying to achieve new things in security research out there, Spectrum: https://spectrum-os.org/ https://spectrum-os.org/ > I'm surprised at the amount of effort people are expending for just this feature with no additional security. NixOS (and Nix more generally) doesn't have a singular feature like that driving its usage or development forward, even though rollbacks is a really nice one that will often come to mind when you ask NixOS users what they like about running NixOS. But if I had to name contenders for the top one or two 'biggest differentiators' from other tools/paradigms that let you achieve similar effects, like other configuration management systems or copy-on-write filesystem snapshotting, I'd say it's the totalizing way that NixOS integrates those features. Because NixOS generates the configuration of the whole system, it gets to avoid having to inspect most of the system's state, and generally handles the bits of transitioning between configurations that do require inspecting and reasoning about the state of the system quickly and pretty well. There's just a smaller surface area there. Similarly, you're just more likely to be able to easily roll back with NixOS because those features are built into all of the normal system/package management operations, and leveraging those things is generally the path of least resistance to changing the system. You end up being able to count on them more 'by default'— you're much less likely to make an important change and have a gap without a snapshot. The garbage collection system makes clearing the unused ones data easier (imo). The general reproducibility also gives you multiple sort of layers of intervention for rolling back— even if you do collect all your past generations, your version control system becomes another reliable way of 'rolling back'. Both of those ways of going back and forth through iterations of your configuration can be further combined with similar interventions at other layers, like dotfile management via Home Manager, snapshotting filesystems for unmanaged files, selective persistence via something like Impermanence, etc. These things can add up a system where the kind of ad-hoc changes that might leak through your state management tools (snapper, etckeeper, dotfile management, etc.) become a radical departure from the way you regularly work. Another differentiator here is maybe the generality: when you Nixify, you sometimes have to do a lot of work up front just to get things working on any deployment target, but the marginal work to go from a NixOS setup to some other kind— generating identical container images, preparing a VM for local use, running your config on macOS, partially sharing your desktop configuration with a server, letting a friend or colleague experiment with or debug your exact setup, preparing an AMI, etc.— is lower, and decreases with each further investment you make in the Nix universe. Different aspects of that inevitably end up being valuable, impressive, or delightful to different users. Taken alone, none of them might seem incomparably compelling over alternative approaches.
fbdab103 3y ago
>...but it does remove any fear of installing software overwriting files you wanted, including OS level (and kernel) upgrades. I guess that is nice to know my Python installation cannot be deleted by malware, but I am more concerned about a process stealing my .ssh keys.
- arjvik 3y ago
  Honestly, exactly recreating my byzantine python installation(s) would probably be impossible [0] (In all seriousness, after switching to Arch I've actually solved this problem by simply installing python packages I need from the AUR) [0]: https://xkcd.com/1987/ https://xkcd.com/1987/

cookiengineer 3y ago

I can recommend reading the systemd manual entries (e.g. man systemd.exec). SystemD meanwhile has a lot of options for managing a seccomp based sandbox, e.g. various protect options for the filesystem, mounting critical things as read-only, simulating a chroot with its own fake-root user etc.pp. You can also manage the capabilities of a binary from there, so it's actually integrated down the kernel stack. However, as you mentioned, the lack of an official "profile database" for common packages/software makes it just as useless as the other tools. I wish we had a repo where all the things come together and people can just do something like "autosandbox apache2" and it will do the rest.

throwing_away 3y ago
Thanks. I'm learning about this today and I'm beginning to suspect all the extra isolation software is not really useful if you configure AppArmor and SystemD properly per service. The space between "full virtual machine" and "unix permission model" is vast and confusing. I would have thought that because everything is hashed on nix, it would be trivial to spin up full "virtual machines" without consuming mountains of disk space, but that does not seem to be an option.
progval 3y ago
Firejail does this. The profile database is the two "profile" directories in https://github.com/netblue30/firejail/tree/master/etc https://github.com/netblue30/firejail/tree/master/etc
D_Sam 3y ago
Sorry… I see no other way to contact you. I saw here in one of your previous comments that you were able to put 32gb of memory on a T440p… can you tell how!? If possible please dm me. Thanks.

nextos 3y ago

> If you want a sandbox on NixOS, your options are still [...] Exactly, I have brought this up in the Nix forum a few times, but developers don't seem to see the value. Really frustrating and a bit ironic given that sandboxing should be quite appealing to a crowd that is attracted to Nix. Guix does implement some a variety of sandboxing options.

paulddraper 3y ago

NixOS isn't a security solution. If you expect that, you will be disappointed. It's isolated for good actors.