6 ms·

This seems to only address a few of the nine threats to the software supply chain, mainly "(D) External build parameters" and maybe the content-addressable stor

by cyrnel 1y ago

This seems to only address a few of the nine threats to the software supply chain, mainly "(D) External build parameters" and maybe the content-addressable storage addresses some of the distribution phase threats: https://slsa.dev/spec/v1.1/threats https://slsa.dev/spec/v1.1/threats There are still many other ways that a dependency can be exploited before or after the build phase.

jchw 1y ago
Nix doesn't, can't, and will obviously never be able to audit your dependencies, but what it can do is give you a way in which you can audit everything byte-for-byte and end-to-end from input to output. In most architectures it is pretty hard to even get to this point because there is no rigorous tracking of dependencies and side-effects; e.g. if your builds are not sandboxed from the network, how can you be sure that the inputs you audited really account for all of the inputs to a build? Nix has a (complete) answer for that, among other things.
- transpute 1y ago
  Debian reproducible builds, Guix, StageX and Yocto/OpenEmbedded have also worked in this area.
  - lillecarl 1y ago
    Yocto is the worst thing over ever worked with. Its just a pile of dirt on top of a pile of dirt
    - transpute 1y ago
      > pile of dirt on top of a pile of dirt Infinite flexibility for reproducible commercial packaging of dirt permutations. Packaging system pain exists at the border of chaos and simulated order. There are useful concepts in Yocto but they were never formalized in academic papers, unlike some build systems for Haskell. There are packaging nuances encoded in bitbake recipes that will likely die there because they work "enough", instead of being further studied for long term lessons. Given that shellcheck is written in Haskell, it might be an interesting academic exercise to write a Haskell replacement for bitbake, which converts bitbake recipes (shell+python) into something more maintainable.
    - xyzzy_plugh 1y ago
      Yocto is significantly better than what proceeded it. I could fill a book on ways it has pushed the envelope for embedded development. Is Yocto a steaming pile? Yes absolutely. But it remains a means to an end. I'd struggle to accomplish with Nix what can be accomplished out of the box with Yocto. Now that being said I'd certainly try and use Nix over Yocto any day of the week. I remain optimistic and that the two will converge and it will more or less not matter, though I'd much prefer Nix to consume embedded development than Yocto become Nix-pilled.
      - lillecarl 1y ago
        I got NixOS running on a NXP board we were using at my last employer and Nix is a blessing if your device is juicy enough to run systemd. If I were doing it again I'd reach for raspberry instead since they're supported everywhere. If the workload can run on it that is.
  - jchw 1y ago
    Reproducible builds are adjacent, but ultimately orthogonal, to what Nix does. Reproducible builds provide strong evidence that a given set of inputs were used to produce a specific output. A lot of the work to be done here by each of these individual projects is beneficial to the entire ecosystem, including Nix. A lot of it is just fixing bugs and removing accidental non-determinism from builds. The main value this provides is that it allows a third-party to verify with relatively good certainty that binaries provided by some entity match the expected source code. Nix provides a hermetic environment to build software with every single input from the bootstrap seed to the final build fully accounted for. Builds can't access the Internet or the host filesystem, they can only access files from their inputs. Impurities must go through the FODs, and the outputs from FODs have to match a prerecorded cryptographic hash, so they must be fully bit-reproducible or the build fails. When you have a reproducible Nix derivation, it is a strict superset of a reproducible package virtually everywhere else, because you can have a very high assurance that you can know and can audit individually every input to the derivation. This is useful for both auditing and reproducible builds. Reproducible builds are important, but reproducible builds alone are not a panacea. They obviously don't tell you if your source code is free of defects, accidental or otherwise, and neither does Nix. Still, Nix does something basically nothing else does, by making the entire build process fully hermetic. (Guix, being inspired by Nix, is, as far as I know, roughly the same, although Guix has put more effort into the bootstrap and package reproducibility. Still, Guix and Nix stand in a class of their own as far as usefulness to supply chain security go, even if they probably won't fit neatly into the compliance theater version of supply chain security.)
    - transpute 1y ago
      Thanks for the detailed explanation. Yocto has offline builds, but is missing host filesystem isolation. Are you familiar with StageX, https://codeberg.org/stagex/stagex/#comparison https://codeberg.org/stagex/stagex/#comparison? There's a comparison chart on that page which claims that Nix(OS?) is not fully reproducible. It would be useful to know which subset, if any, is not reproducible.
      - jchw 1y ago
        It is correct that NixOS is not fully reproducible. Regarding Nix vs NixOS vs Nixpkgs: - Nix is the name of the programming language (that Nix derivations are written in) and the package manager/build tool. (Though at times the term 'Nix' is just used to refer to the broader ecosystem including NixOS and Nixpkgs.) - NixOS is an operating system that is built on top of Nix derivations. - Nixpkgs is a giant repository of Nix derivations, including NixOS (NixOS and Nixpkgs used to be separate, but they were merged together in history.) The reason why NixOS+Nixpkgs are not fully reproducible, despite all of the guarantees Nix gives you, is simply because there are derivations with non-determinism during the build process. An example of how this might play out is that you could wind up with the order in which operations complete in a parallel build process somehow getting encoded into the final package. Unfortunately, I don't think there's funding or a ton of interest going into improving the reproducibility of NixOS at the moment, so progress towards squashing reproducibility issues has been slow for a while. You can see relatively up-to-date progress on getting the install media fully reproducible here: https://reproducible.nixos.org/nixos-iso-minimal-runtime/ https://reproducible.nixos.org/nixos-iso-minimal-runtime/ StageX also correctly points out that the Nix bootstrap is inferior to some of the more extreme reproducibility projects. The Nix bootstrap is fairly large, unfortunately. The Guix team has put a substantial amount of effort into minimizing the bootstrap seed and reproducibility of packages. The Nixpkgs bootstrap seed (for my machine, anyway) is currently 27 MiB. The GuixSD bootstrap seed is, I believe, 357 bytes, which is a stunning accomplishment. StageX considers NixOS trust to be centralized and GuixSD trust to be distributed; this is likely because of the Hydra binary cache which Nix is typically configured to trust by default. You can turn off the Hydra cache to remove this centralized entity, at the cost of obviously needing to build almost everything from scratch. I'm not sure what "distributed" trust actually means here, versus "decentralized". StageX uses OCI image building as a base. It also doesn't seem to talk about sandboxing anywhere, so it is presumed that StageX is using Dockerfile OCI builds as their only sandboxing, which still allows Internet access. Having Internet access during builds is convenient, but it makes it pretty hard to guarantee that all inputs are accounted for. Their Rust example is pretty interesting: > RUN ["cargo", "add", "regex"] There's nothing inherently wrong about this, but despite all of the effort to make the base StageX OCI images reproducible, if you were to build this exact same OCI image months apart, you would presumably be liable to get different results here: you could, for example, get an entirely different version of the regex crate. With Nix, if you make a derivation to build a Rust package, you have to account for the Cargo dependencies in the build, as Nix builds aren't allowed to access the Internet, with the exception of fixed-output derivations. While this doesn't result in Nix derivations being bit-for-bit reproducible, it does ensure that every (external) input is bit-for-bit identical across builds to the same exact derivation, something you can't really easily achieve without a custom build tool like Nix or Guix. If it were possible to sneak an impurity into a Nix build, it is likely a CVE. (There are some exceptions on macOS due to limitations in Darwin sandboxing, but on Linux I believe this holds true. None of the exceptions would make it possible to easily accidentally introduce impurities on macOS, though; you'd pretty much need to do it on purpose.) Even that aside, StageX uses the same impressive 357 byte bootstrap seed as GuixSD, so it is pretty cool for what it does. It's just a bit lower in scope than Nixpkgs and GuixSD. Nixpkgs is probably the largest single software repository ever built with over 100,000 packages, all of which having to follow this schema of hermetic builds.

by cyrnel 1y ago

jchw 1y ago

Nix doesn't, can't, and will obviously never be able to audit your dependencies, but what it can do is give you a way in which you can audit everything byte-for-byte and end-to-end from input to output. In most architectures it is pretty hard to even get to this point because there is no rigorous tracking of dependencies and side-effects; e.g. if your builds are not sandboxed from the network, how can you be sure that the inputs you audited really account for all of the inputs to a build? Nix has a (complete) answer for that, among other things.

transpute 1y ago
Debian reproducible builds, Guix, StageX and Yocto/OpenEmbedded have also worked in this area.
- lillecarl 1y ago
  Yocto is the worst thing over ever worked with. Its just a pile of dirt on top of a pile of dirt
  - transpute 1y ago
    > pile of dirt on top of a pile of dirt Infinite flexibility for reproducible commercial packaging of dirt permutations. Packaging system pain exists at the border of chaos and simulated order. There are useful concepts in Yocto but they were never formalized in academic papers, unlike some build systems for Haskell. There are packaging nuances encoded in bitbake recipes that will likely die there because they work "enough", instead of being further studied for long term lessons. Given that shellcheck is written in Haskell, it might be an interesting academic exercise to write a Haskell replacement for bitbake, which converts bitbake recipes (shell+python) into something more maintainable.
  - xyzzy_plugh 1y ago
    Yocto is significantly better than what proceeded it. I could fill a book on ways it has pushed the envelope for embedded development. Is Yocto a steaming pile? Yes absolutely. But it remains a means to an end. I'd struggle to accomplish with Nix what can be accomplished out of the box with Yocto. Now that being said I'd certainly try and use Nix over Yocto any day of the week. I remain optimistic and that the two will converge and it will more or less not matter, though I'd much prefer Nix to consume embedded development than Yocto become Nix-pilled.
    - lillecarl 1y ago
      I got NixOS running on a NXP board we were using at my last employer and Nix is a blessing if your device is juicy enough to run systemd. If I were doing it again I'd reach for raspberry instead since they're supported everywhere. If the workload can run on it that is.
- jchw 1y ago
  Reproducible builds are adjacent, but ultimately orthogonal, to what Nix does. Reproducible builds provide strong evidence that a given set of inputs were used to produce a specific output. A lot of the work to be done here by each of these individual projects is beneficial to the entire ecosystem, including Nix. A lot of it is just fixing bugs and removing accidental non-determinism from builds. The main value this provides is that it allows a third-party to verify with relatively good certainty that binaries provided by some entity match the expected source code. Nix provides a hermetic environment to build software with every single input from the bootstrap seed to the final build fully accounted for. Builds can't access the Internet or the host filesystem, they can only access files from their inputs. Impurities must go through the FODs, and the outputs from FODs have to match a prerecorded cryptographic hash, so they must be fully bit-reproducible or the build fails. When you have a reproducible Nix derivation, it is a strict superset of a reproducible package virtually everywhere else, because you can have a very high assurance that you can know and can audit individually every input to the derivation. This is useful for both auditing and reproducible builds. Reproducible builds are important, but reproducible builds alone are not a panacea. They obviously don't tell you if your source code is free of defects, accidental or otherwise, and neither does Nix. Still, Nix does something basically nothing else does, by making the entire build process fully hermetic. (Guix, being inspired by Nix, is, as far as I know, roughly the same, although Guix has put more effort into the bootstrap and package reproducibility. Still, Guix and Nix stand in a class of their own as far as usefulness to supply chain security go, even if they probably won't fit neatly into the compliance theater version of supply chain security.)
  - transpute 1y ago
    Thanks for the detailed explanation. Yocto has offline builds, but is missing host filesystem isolation. Are you familiar with StageX, https://codeberg.org/stagex/stagex/#comparison https://codeberg.org/stagex/stagex/#comparison? There's a comparison chart on that page which claims that Nix(OS?) is not fully reproducible. It would be useful to know which subset, if any, is not reproducible.
    - jchw 1y ago
      It is correct that NixOS is not fully reproducible. Regarding Nix vs NixOS vs Nixpkgs: - Nix is the name of the programming language (that Nix derivations are written in) and the package manager/build tool. (Though at times the term 'Nix' is just used to refer to the broader ecosystem including NixOS and Nixpkgs.) - NixOS is an operating system that is built on top of Nix derivations. - Nixpkgs is a giant repository of Nix derivations, including NixOS (NixOS and Nixpkgs used to be separate, but they were merged together in history.) The reason why NixOS+Nixpkgs are not fully reproducible, despite all of the guarantees Nix gives you, is simply because there are derivations with non-determinism during the build process. An example of how this might play out is that you could wind up with the order in which operations complete in a parallel build process somehow getting encoded into the final package. Unfortunately, I don't think there's funding or a ton of interest going into improving the reproducibility of NixOS at the moment, so progress towards squashing reproducibility issues has been slow for a while. You can see relatively up-to-date progress on getting the install media fully reproducible here: https://reproducible.nixos.org/nixos-iso-minimal-runtime/ https://reproducible.nixos.org/nixos-iso-minimal-runtime/ StageX also correctly points out that the Nix bootstrap is inferior to some of the more extreme reproducibility projects. The Nix bootstrap is fairly large, unfortunately. The Guix team has put a substantial amount of effort into minimizing the bootstrap seed and reproducibility of packages. The Nixpkgs bootstrap seed (for my machine, anyway) is currently 27 MiB. The GuixSD bootstrap seed is, I believe, 357 bytes, which is a stunning accomplishment. StageX considers NixOS trust to be centralized and GuixSD trust to be distributed; this is likely because of the Hydra binary cache which Nix is typically configured to trust by default. You can turn off the Hydra cache to remove this centralized entity, at the cost of obviously needing to build almost everything from scratch. I'm not sure what "distributed" trust actually means here, versus "decentralized". StageX uses OCI image building as a base. It also doesn't seem to talk about sandboxing anywhere, so it is presumed that StageX is using Dockerfile OCI builds as their only sandboxing, which still allows Internet access. Having Internet access during builds is convenient, but it makes it pretty hard to guarantee that all inputs are accounted for. Their Rust example is pretty interesting: > RUN ["cargo", "add", "regex"] There's nothing inherently wrong about this, but despite all of the effort to make the base StageX OCI images reproducible, if you were to build this exact same OCI image months apart, you would presumably be liable to get different results here: you could, for example, get an entirely different version of the regex crate. With Nix, if you make a derivation to build a Rust package, you have to account for the Cargo dependencies in the build, as Nix builds aren't allowed to access the Internet, with the exception of fixed-output derivations. While this doesn't result in Nix derivations being bit-for-bit reproducible, it does ensure that every (external) input is bit-for-bit identical across builds to the same exact derivation, something you can't really easily achieve without a custom build tool like Nix or Guix. If it were possible to sneak an impurity into a Nix build, it is likely a CVE. (There are some exceptions on macOS due to limitations in Darwin sandboxing, but on Linux I believe this holds true. None of the exceptions would make it possible to easily accidentally introduce impurities on macOS, though; you'd pretty much need to do it on purpose.) Even that aside, StageX uses the same impressive 357 byte bootstrap seed as GuixSD, so it is pretty cool for what it does. It's just a bit lower in scope than Nixpkgs and GuixSD. Nixpkgs is probably the largest single software repository ever built with over 100,000 packages, all of which having to follow this schema of hermetic builds.