9 ms·

Note that NixOS and reproducible builds did not detect the xz backdoor, and in fact NixOS shipped the malicious builds of xz (though they didn't do anything bec

Note that NixOS and reproducible builds did not detect the xz backdoor, and in fact NixOS shipped the malicious builds of xz (though they didn't do anything because the malware didn't target NixOS): > I am a NixOS developer and I was surprised when the backdoor was revealed to see that the malicious version of xz had ended up being distributed to our users. As always theory and reality are different, and the thing that made xz possible was never a technical vulnerability with a technical solution—xz was possible because of a meatspace exploit. We as a community are very very bad at recognizing that you can't always just patch meatspace with better software.

jjmarr 2y ago
I still like the blogpost, because NixOS bills itself as a technical solution to prevent build artifacts that are decoupled from the source code (i.e. not reproducible), and the xz backdoor was hidden in build artifacts.
- lolinder 2y ago
  Yeah, it's a good blog post in part because it gets into the details of how it was possible that this vulnerability made it into NixOS, which purports to solve the problem. Also, I'm not a NixOS critic either: I'm writing this from NixOS! I just don't think there's such a thing as a security cure-all as long as humans are in the loop anywhere.
- crote 2y ago
  Sure, but you could achieve the same thing by requiring that the build artifacts are generated by a Github Actions runner.
  - tomjakubowski 2y ago
    That would only make the xz attack harder, not make it impossible. Just add some step to the action run that fetches blobs from an internet resource you control, then swap out the blobs for malicious ones.
- deleted 2y ago
  [deleted]
nextos 2y ago
> NixOS and reproducible builds did not detect the xz backdoor Nix declarativeness is quite useful to increase protection against exploits in a number of ways. Unfortunately, there is still a lot of untapped potential. My number one priority would be to implement fine-grained ephemeral containers. Guix has these already. This would make it convenient to run every single process with restricted privileges, including no access to ~/, except those directories that are needed by the task. That would prevent e.g. a rogue pip package from stealing SSH keys. Still, I think the xz backdoor did not work on NixOS because its unusual non FHS-compliant filesystem structure.
- ratorx 2y ago
  You don’t even need to run in a container for this. It’s possible to do this entirely in systemd service configuration. The easiest way is just to have separate user for every service and reduce stuff running as root. You can also restrict filesystem access, network access and even syscall access (although some of this may be implemented as a container under the hood). Unfortunately, this wouldn’t help with the xz vulnerability because the SSH server is the one loading the compromised library in that case (indirectly). Since SSH itself needs to have access to the private keys, it’s not really easy to secure it against vulnerabilities in the library it loads itself. On the flip side, unless the vulnerability is in one of the important binaries/shared libraries, the amount of damage it can cause it probably quite contained with simply having good user isolation. Nix can make this analysis really simple (because of explicitly specified dependencies), so you can crack down on critical dependencies a lot more easily.
  - nextos 2y ago
    > It’s possible to do this entirely in systemd service configuration Sure, but I think that leaves out many use cases. What if I want to e.g. start a Python shell that has access to certain directories, and nothing else, including no network access? Nix provides a good way of doing that for common use cases, as it has decent support for Firejail. But I would like something like Guix containers, which is convenient for any ad hoc use case. This greatly reduces any security threat. It's a poor-man's QubesOS.
sirspamalot101 2y ago
[flagged]
sirspamalot103 2y ago
[dead]
sirspamalot105 2y ago
[dead]
sirspamalot107 2y ago
[flagged]
sirspamalot112 2y ago
[dead]
banana_dick_1 2y ago
[flagged]
nebulous1 2y ago
Right. The title seemed to be suggesting that the Nix way of doing things might have detected the backdoor. It's actually intending to suggest ways that Nix could be changed in order to detect the backdoor.
dist-epoch 2y ago
The reason is a bit funny - the NixOS bootstrap downloads it's source code which is a xz compressed tarball.
johnea 2y ago
> As always theory and reality are different Thank you very much for citing that! along with highlighting the fact that the exploit was in fact, not detected by reproducible builds prior to other means of discovery. In recent times, actual reality is often maligned when compared to how people feel about objective reality and how it meshes with their individual value systems. I have personal values too, but I don't hold the opinion that actual reality is less significant than how I feel about it. It's not a popular perspective 8-/ I've always liked to say: The difference between theory and reality is that, in theory they're the same, and in reality they're not. I hope the realization that the reproducible builds of NixOS _could_ have detected the xz exploit, but didn't, will lead to new advances in the analysis of those reproducible builds to detect other exploits sooner in the future.

jjmarr 2y ago

I still like the blogpost, because NixOS bills itself as a technical solution to prevent build artifacts that are decoupled from the source code (i.e. not reproducible), and the xz backdoor was hidden in build artifacts.

lolinder 2y ago
Yeah, it's a good blog post in part because it gets into the details of how it was possible that this vulnerability made it into NixOS, which purports to solve the problem. Also, I'm not a NixOS critic either: I'm writing this from NixOS! I just don't think there's such a thing as a security cure-all as long as humans are in the loop anywhere.
crote 2y ago
Sure, but you could achieve the same thing by requiring that the build artifacts are generated by a Github Actions runner.
- tomjakubowski 2y ago
  That would only make the xz attack harder, not make it impossible. Just add some step to the action run that fetches blobs from an internet resource you control, then swap out the blobs for malicious ones.
deleted 2y ago
[deleted]

nextos 2y ago

> NixOS and reproducible builds did not detect the xz backdoor Nix declarativeness is quite useful to increase protection against exploits in a number of ways. Unfortunately, there is still a lot of untapped potential. My number one priority would be to implement fine-grained ephemeral containers. Guix has these already. This would make it convenient to run every single process with restricted privileges, including no access to ~/, except those directories that are needed by the task. That would prevent e.g. a rogue pip package from stealing SSH keys. Still, I think the xz backdoor did not work on NixOS because its unusual non FHS-compliant filesystem structure.

ratorx 2y ago
You don’t even need to run in a container for this. It’s possible to do this entirely in systemd service configuration. The easiest way is just to have separate user for every service and reduce stuff running as root. You can also restrict filesystem access, network access and even syscall access (although some of this may be implemented as a container under the hood). Unfortunately, this wouldn’t help with the xz vulnerability because the SSH server is the one loading the compromised library in that case (indirectly). Since SSH itself needs to have access to the private keys, it’s not really easy to secure it against vulnerabilities in the library it loads itself. On the flip side, unless the vulnerability is in one of the important binaries/shared libraries, the amount of damage it can cause it probably quite contained with simply having good user isolation. Nix can make this analysis really simple (because of explicitly specified dependencies), so you can crack down on critical dependencies a lot more easily.
- nextos 2y ago
  > It’s possible to do this entirely in systemd service configuration Sure, but I think that leaves out many use cases. What if I want to e.g. start a Python shell that has access to certain directories, and nothing else, including no network access? Nix provides a good way of doing that for common use cases, as it has decent support for Firejail. But I would like something like Guix containers, which is convenient for any ad hoc use case. This greatly reduces any security threat. It's a poor-man's QubesOS.

sirspamalot101 2y ago

[flagged]

sirspamalot103 2y ago

[dead]

sirspamalot105 2y ago

[dead]

sirspamalot107 2y ago

[flagged]

sirspamalot112 2y ago

[dead]

banana_dick_1 2y ago

[flagged]

nebulous1 2y ago

Right. The title seemed to be suggesting that the Nix way of doing things might have detected the backdoor. It's actually intending to suggest ways that Nix could be changed in order to detect the backdoor.

dist-epoch 2y ago

The reason is a bit funny - the NixOS bootstrap downloads it's source code which is a xz compressed tarball.

johnea 2y ago

> As always theory and reality are different Thank you very much for citing that! along with highlighting the fact that the exploit was in fact, not detected by reproducible builds prior to other means of discovery. In recent times, actual reality is often maligned when compared to how people feel about objective reality and how it meshes with their individual value systems. I have personal values too, but I don't hold the opinion that actual reality is less significant than how I feel about it. It's not a popular perspective 8-/ I've always liked to say: The difference between theory and reality is that, in theory they're the same, and in reality they're not. I hope the realization that the reproducible builds of NixOS _could_ have detected the xz exploit, but didn't, will lead to new advances in the analysis of those reproducible builds to detect other exploits sooner in the future.