5 ms·

It’s almost impossible to achieve any sort of reproducibility when all packages are built from source. Portage is a great tool, but I always thought it was bet

It’s almost impossible to achieve any sort of reproducibility when all packages are built from source. Portage is a great tool, but I always thought it was better suited to build packages and operating systems as opposed being an end user package management solution.

KirinDave 9y ago
I've learned recently that even using Nix when building from source is something of a false hope of reproducibility. Many packages use the fetchgit function and fetchgit violates the contract.
- Filligree 9y ago
  fetchgit does no such thing. To use it, you need to specify the git revision and the hash of the output; it's a constant-output derivation just like fetchurl. Reproducbility is still a difficult problem, but that isn't one of them, and Nix comes far closer than Gentoo.
  - KirinDave 9y ago
    https://twitter.com/shlevy/status/956313877800566784?ref_src=twcamp%5Ecopy%7Ctwsrc%5Eandroid%7Ctwgr%5Ecopy%7Ctwcon%5E7090%7Ctwterm%5E0 https://twitter.com/shlevy/status/956313877800566784?ref_src...
    - mst 9y ago
      I do hope somebody either makes that start warning, then removes it, or at least adds it to a linter of some sort.
korethr 9y ago
If I am correctly recalling a conversation I had with Gentoo founder Daniel Robbins on IRC a couple years back, Portage was originally intended as a tool for building binary packages, not necessarily the end-user package management. As for reproducibility when building from source, that depends on your practices. Is every box a special snowflake with it's own USE flags and package set? Well then yes, good luck with reproducibility. Or are you standardizing your targets and building one package to deploy to them all?
- kitotik 9y ago
  Build a gentoo system each day for seven days on the same hardware with the same configuration(package versions, USE flags, etc) and see how well they match up.
  - korethr 9y ago
    Okay, I'm tempted now, partly out of a someone-might-be-wrong-on-the-Internet cussedness, and partly out of curiosity. By what metrics would you propose measuring how consistent the systems are or are not?
    - TheDong 9y ago
      Binpkgs output from builds on the same machine are often bit-for-bit identical. It, of course, depends on the project; if the ./configure script drops the current date in source code, it'll be different, but that doesn't happen often.
  - laumars 9y ago
    I have done this. I used to manage a fleet of 50 servers with everything built from source aside the base Linux install and build utilities. They were all identical.
darkr 9y ago
Hard, but not impossible: https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds

KirinDave 9y ago

I've learned recently that even using Nix when building from source is something of a false hope of reproducibility. Many packages use the fetchgit function and fetchgit violates the contract.

Filligree 9y ago
fetchgit does no such thing. To use it, you need to specify the git revision and the hash of the output; it's a constant-output derivation just like fetchurl. Reproducbility is still a difficult problem, but that isn't one of them, and Nix comes far closer than Gentoo.
- KirinDave 9y ago
  https://twitter.com/shlevy/status/956313877800566784?ref_src=twcamp%5Ecopy%7Ctwsrc%5Eandroid%7Ctwgr%5Ecopy%7Ctwcon%5E7090%7Ctwterm%5E0 https://twitter.com/shlevy/status/956313877800566784?ref_src...
  - mst 9y ago
    I do hope somebody either makes that start warning, then removes it, or at least adds it to a linter of some sort.

korethr 9y ago

If I am correctly recalling a conversation I had with Gentoo founder Daniel Robbins on IRC a couple years back, Portage was originally intended as a tool for building binary packages, not necessarily the end-user package management. As for reproducibility when building from source, that depends on your practices. Is every box a special snowflake with it's own USE flags and package set? Well then yes, good luck with reproducibility. Or are you standardizing your targets and building one package to deploy to them all?

kitotik 9y ago
Build a gentoo system each day for seven days on the same hardware with the same configuration(package versions, USE flags, etc) and see how well they match up.
- korethr 9y ago
  Okay, I'm tempted now, partly out of a someone-might-be-wrong-on-the-Internet cussedness, and partly out of curiosity. By what metrics would you propose measuring how consistent the systems are or are not?
  - TheDong 9y ago
    Binpkgs output from builds on the same machine are often bit-for-bit identical. It, of course, depends on the project; if the ./configure script drops the current date in source code, it'll be different, but that doesn't happen often.
- laumars 9y ago
  I have done this. I used to manage a fleet of 50 servers with everything built from source aside the base Linux install and build utilities. They were all identical.

darkr 9y ago

Hard, but not impossible: https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds