4 ms·

It's early on the weekend but the question I predict is "What made Nix/Guix unsuitable as a part in this?" The technology keyword SEO is strong around here! H

It's early on the weekend but the question I predict is "What made Nix/Guix unsuitable as a part in this?" The technology keyword SEO is strong around here! However if you've already documented pros/cons such a list could provide constructive direction for those projects.

woodruffw 3y ago
> It's early on the weekend but the question I predict is "What made Nix/Guix unsuitable as a part in this?" I'm not sure I understand: Homebrew is a package manager in its own right, so it doesn't make a lot of sense for it to retrofit an entire other packaging ecosystem (Nix or Guix) on top of it. (I don't know those ecosystems particularly well, but my understanding is that neither has code signing/build provenance either -- for Nix at least my understanding is that there are no remote builds in the first place, so build provenance isn't relevant.)
- password4321 3y ago
  Thanks for answering my question; I apologize for not explaining it very well. You are creating CI/CD infrastructure to build binaries for Homebrew. Per my limited understanding, Nix/Guix will someday build reproducible binaries, but it didn't offer value for you here at the end of 2023. The reasons why might help the projects be more useful to those with similar goals in the future.
  - woodruffw 3y ago
    > You are creating CI/CD infrastructure to build binaries for Homebrew. Not exactly: Homebrew already has extensive, mature infrastructure for building binary distributions (in Homebrew nomenclature, "bottles"). The point of this work is solely to offer cryptographic attestations for those bottles, namely attesting that they're being produced on Homebrew's (GitHub Actions-based) CI/CD. Reproducibility would be nice, but it isn't a requirement for this work. Nix and Guix would possibly help with that, but adding it would be (1) a significant engineering effort, with no clearly defined borders, and (2) would not solve the actual problem at hand. Edit: I realized the above doesn't offer a "why." The "why" for all of this is improved integrity: it adds another hurdle for an attacker looking to download and install malicious bottles through Homebrew. Additionally, it makes it easier to host a verifiable mirror of homebrew-core's bottles.
    - deleted 3y ago
      [deleted]
- tomberek 3y ago
  Nix (and by extension Guix) has had remote builds and signed build provenance for nearly 2 decades. It has also had a story to deal with both content-addrrssed and input-addressed (this is our term for the "stable identifier" for a workflow mentioned above). I can understand not using Nix, but please take a look at how we've solved some of these problems, and our mistakes. There is no need to reinvent the wheel blindly. Feel free to reach out if interested.
  - woodruffw 3y ago
    > Nix (and by extension Guix) has had remote builds and signed build provenance for nearly 2 decades. Where is this documented? When I search for "Nix code signing" I mostly get issue reports for people experiencing breakage due to macOS's (separate) code signing requirements. Edit: To be clear, I like Nix, and I find many of its properties appealing. At the same time, it's not clear to me that Homebrew has too much to glean here: Homebrew isn't trying to be a totalizing package/system state management solution, and has historically leaned heavily on automation and external services (GHA, GHCR, etc.). These are design decisions with tradeoffs; they've also informed the proposed build provenance design substantially.
    - tomberek 3y ago
      There is a naming problem here where "Nix" is often easily confused with the state management pieces (home-manager, NixOS, etc). The underlying Store abstraction provides the concepts of various kinds of addressing. This is independent of the Nix language and we are currently trying to expose each of these layers more directly so it can be re-used in other contexts. The piece that might be of interest to you is the interaction between content-addressed and input-addressed content in light of build systems+software+compilers that are not bit-reproducible. There is also the bookkeeping to ensure auditability by third-parties; anyone else can run the same builds and expect the same outcome, or close enough that the differences become easy to find and report upstream (https://r13y.com/ https://r13y.com/), but the system doesn't require bit-reproducibility from day1 because that would be impractical. Of course substituting from a cache is exact, made possible by the signatures. Remote builds: https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html https://nixos.org/manual/nix/stable/advanced-topics/distribu... Signing (i'll amend my claim above to be only 1 decade). These are either signatures of a CA path, or for input-addressed things it can be seen as a claim by the signer that the producing build recipe (we call it "derivation") has this particular binary output. - https://nixos.org/manual/nix/stable/command-ref/nix-store/generate-binary-cache-key.html https://nixos.org/manual/nix/stable/command-ref/nix-store/ge... - https://nixos.org/manual/nix/stable/advanced-topics/post-build-hook.html?highlight=keys#set-up-a-signing-key https://nixos.org/manual/nix/stable/advanced-topics/post-bui... - https://nixos.org/manual/nix/stable/command-ref/conf-file.html?highlight=trusted-public-keys#conf-require-sigs https://nixos.org/manual/nix/stable/command-ref/conf-file.ht... - https://nixos.org/manual/nix/stable/command-ref/conf-file.html?highlight=trusted-public-keys#conf-trusted-public-keys https://nixos.org/manual/nix/stable/command-ref/conf-file.ht... - example: see the Signatures line for an example that this this specific provenance produced this specific binary output: https://trusted-friendly-sesame.glitch.me/view.html?cache_base=https%3A%2F%2Fcache.nixos.org&hash=7ghhnlwla2mddkg7hgqa5v0sr8g5hga8 https://trusted-friendly-sesame.glitch.me/view.html?cache_ba... or https://cache.nixos.org/7ghhnlwla2mddkg7hgqa5v0sr8g5hga8.narinfo https://cache.nixos.org/7ghhnlwla2mddkg7hgqa5v0sr8g5hga8.nar...
stakhanov 3y ago
You seem to be throwing in some thoughts while simultaneously attributing them to an unidentified third party and distancing yourself from them, which is a bit confusing. My two cents' worth: Just because a technology has reached buzzword status in a given social cluster doesn't impose a duty on everyone in that cluster to explain themselves for not using it. Personally, I'm a huge fan of brew, while being not such a huge fan of nix/guix, despite the fact that nix/guix is a buzzword around here while brew is not.
- password4321 3y ago
  > You seem to be throwing in some thoughts while simultaneously attributing them to an unidentified third party and distancing yourself from them, which is a bit confusing. True. "If the crowd was here they might want to know why their favorite hammer didn't help" is more a best guess to generate helpful documentation should this survive on the front page until tomorrow. I did not intend to create an additional burden on anyone. I appreciated the offer to answer questions, and the mic was open while they were around. Thank you for sharing your experience with Mac package managers. I used brew but switched to macports the second time I forgot to disable "auto update the world" as Apple+brew left my comfortable old macOS version behind (I use older hardware, I don't track macOS release dates, and brew uninstalled working software before failing to build the updated version). I have been too terrified to try Nix on Mac due to the old "I couldn't uninstall" issues.
  - stakhanov 3y ago
    I actually use linuxbrew and don't use macs at all. Specifically, the way I like to set up my dev environments is to use Gentoo for the root filesystem, but also throw in linuxbrew to balance compile-from-source purism with binary-package pragmatism for packages and usecases where I just don't care as much about where my packages came from and how they were compiled. One of the things I like about brew is that I can just tarball-up /home/linuxbrew/.linuxbrew and if I get myself into a pickle, I can just unpack a tarball in that location and be up-and-running. After I make any meaningful set of changes to the environment that I'm developing against, I make a versioned tarball and keep it for all eternity, that way I can have reasonable hope that if I ever want to run code I wrote a long time ago and haven't used for a long time, I will be able to re-create the development environment that I was using around it.

woodruffw 3y ago

> It's early on the weekend but the question I predict is "What made Nix/Guix unsuitable as a part in this?" I'm not sure I understand: Homebrew is a package manager in its own right, so it doesn't make a lot of sense for it to retrofit an entire other packaging ecosystem (Nix or Guix) on top of it. (I don't know those ecosystems particularly well, but my understanding is that neither has code signing/build provenance either -- for Nix at least my understanding is that there are no remote builds in the first place, so build provenance isn't relevant.)

password4321 3y ago
Thanks for answering my question; I apologize for not explaining it very well. You are creating CI/CD infrastructure to build binaries for Homebrew. Per my limited understanding, Nix/Guix will someday build reproducible binaries, but it didn't offer value for you here at the end of 2023. The reasons why might help the projects be more useful to those with similar goals in the future.
- woodruffw 3y ago
  > You are creating CI/CD infrastructure to build binaries for Homebrew. Not exactly: Homebrew already has extensive, mature infrastructure for building binary distributions (in Homebrew nomenclature, "bottles"). The point of this work is solely to offer cryptographic attestations for those bottles, namely attesting that they're being produced on Homebrew's (GitHub Actions-based) CI/CD. Reproducibility would be nice, but it isn't a requirement for this work. Nix and Guix would possibly help with that, but adding it would be (1) a significant engineering effort, with no clearly defined borders, and (2) would not solve the actual problem at hand. Edit: I realized the above doesn't offer a "why." The "why" for all of this is improved integrity: it adds another hurdle for an attacker looking to download and install malicious bottles through Homebrew. Additionally, it makes it easier to host a verifiable mirror of homebrew-core's bottles.
  - deleted 3y ago
    [deleted]
tomberek 3y ago
Nix (and by extension Guix) has had remote builds and signed build provenance for nearly 2 decades. It has also had a story to deal with both content-addrrssed and input-addressed (this is our term for the "stable identifier" for a workflow mentioned above). I can understand not using Nix, but please take a look at how we've solved some of these problems, and our mistakes. There is no need to reinvent the wheel blindly. Feel free to reach out if interested.
- woodruffw 3y ago
  > Nix (and by extension Guix) has had remote builds and signed build provenance for nearly 2 decades. Where is this documented? When I search for "Nix code signing" I mostly get issue reports for people experiencing breakage due to macOS's (separate) code signing requirements. Edit: To be clear, I like Nix, and I find many of its properties appealing. At the same time, it's not clear to me that Homebrew has too much to glean here: Homebrew isn't trying to be a totalizing package/system state management solution, and has historically leaned heavily on automation and external services (GHA, GHCR, etc.). These are design decisions with tradeoffs; they've also informed the proposed build provenance design substantially.
  - tomberek 3y ago
    There is a naming problem here where "Nix" is often easily confused with the state management pieces (home-manager, NixOS, etc). The underlying Store abstraction provides the concepts of various kinds of addressing. This is independent of the Nix language and we are currently trying to expose each of these layers more directly so it can be re-used in other contexts. The piece that might be of interest to you is the interaction between content-addressed and input-addressed content in light of build systems+software+compilers that are not bit-reproducible. There is also the bookkeeping to ensure auditability by third-parties; anyone else can run the same builds and expect the same outcome, or close enough that the differences become easy to find and report upstream (https://r13y.com/ https://r13y.com/), but the system doesn't require bit-reproducibility from day1 because that would be impractical. Of course substituting from a cache is exact, made possible by the signatures. Remote builds: https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html https://nixos.org/manual/nix/stable/advanced-topics/distribu... Signing (i'll amend my claim above to be only 1 decade). These are either signatures of a CA path, or for input-addressed things it can be seen as a claim by the signer that the producing build recipe (we call it "derivation") has this particular binary output. - https://nixos.org/manual/nix/stable/command-ref/nix-store/generate-binary-cache-key.html https://nixos.org/manual/nix/stable/command-ref/nix-store/ge... - https://nixos.org/manual/nix/stable/advanced-topics/post-build-hook.html?highlight=keys#set-up-a-signing-key https://nixos.org/manual/nix/stable/advanced-topics/post-bui... - https://nixos.org/manual/nix/stable/command-ref/conf-file.html?highlight=trusted-public-keys#conf-require-sigs https://nixos.org/manual/nix/stable/command-ref/conf-file.ht... - https://nixos.org/manual/nix/stable/command-ref/conf-file.html?highlight=trusted-public-keys#conf-trusted-public-keys https://nixos.org/manual/nix/stable/command-ref/conf-file.ht... - example: see the Signatures line for an example that this this specific provenance produced this specific binary output: https://trusted-friendly-sesame.glitch.me/view.html?cache_base=https%3A%2F%2Fcache.nixos.org&hash=7ghhnlwla2mddkg7hgqa5v0sr8g5hga8 https://trusted-friendly-sesame.glitch.me/view.html?cache_ba... or https://cache.nixos.org/7ghhnlwla2mddkg7hgqa5v0sr8g5hga8.narinfo https://cache.nixos.org/7ghhnlwla2mddkg7hgqa5v0sr8g5hga8.nar...

stakhanov 3y ago

You seem to be throwing in some thoughts while simultaneously attributing them to an unidentified third party and distancing yourself from them, which is a bit confusing. My two cents' worth: Just because a technology has reached buzzword status in a given social cluster doesn't impose a duty on everyone in that cluster to explain themselves for not using it. Personally, I'm a huge fan of brew, while being not such a huge fan of nix/guix, despite the fact that nix/guix is a buzzword around here while brew is not.

password4321 3y ago
> You seem to be throwing in some thoughts while simultaneously attributing them to an unidentified third party and distancing yourself from them, which is a bit confusing. True. "If the crowd was here they might want to know why their favorite hammer didn't help" is more a best guess to generate helpful documentation should this survive on the front page until tomorrow. I did not intend to create an additional burden on anyone. I appreciated the offer to answer questions, and the mic was open while they were around. Thank you for sharing your experience with Mac package managers. I used brew but switched to macports the second time I forgot to disable "auto update the world" as Apple+brew left my comfortable old macOS version behind (I use older hardware, I don't track macOS release dates, and brew uninstalled working software before failing to build the updated version). I have been too terrified to try Nix on Mac due to the old "I couldn't uninstall" issues.
- stakhanov 3y ago
  I actually use linuxbrew and don't use macs at all. Specifically, the way I like to set up my dev environments is to use Gentoo for the root filesystem, but also throw in linuxbrew to balance compile-from-source purism with binary-package pragmatism for packages and usecases where I just don't care as much about where my packages came from and how they were compiled. One of the things I like about brew is that I can just tarball-up /home/linuxbrew/.linuxbrew and if I get myself into a pickle, I can just unpack a tarball in that location and be up-and-running. After I make any meaningful set of changes to the environment that I'm developing against, I make a versioned tarball and keep it for all eternity, that way I can have reasonable hope that if I ever want to run code I wrote a long time ago and haven't used for a long time, I will be able to re-create the development environment that I was using around it.