4 ms·

I like the idea a lot. However what's lacking is good package repositories to go with it. Nix has exactly the same issue. A question I have is why don't they u

by norswap 12y ago

I like the idea a lot. However what's lacking is good package repositories to go with it. Nix has exactly the same issue. A question I have is why don't they use the Debian / Fedora repositories? Is there some crucial meta-data missing? Are the build script for .deb / .rpm made in such a way that it is not possible to add multi-version support on top? To clarify, I'm not asking them to interoperate with apt-get/yum, just to use the package repositories as a source.

davexunit 12y ago
>why don't they use the Debian / Fedora repositories? They are fundamentally incompatible. Among other reasons, Debian and Fedora package builds are not reproducible. That is, given the corresponding source code to the package, you're not likely to get the same binary if you build it yourself. Sometimes maintainer uploaded packages may not build at all for someone else, because the binary built on the maintainer's machine relied on some extra software not specified in the build recipe. Nix/Guix are source-based distributions. They build packages in an isolated chroot, in which only the explicit dependencies of a package are available. This means that no user must rely on a server to give them binaries. Any user can opt to build any package on their own system instead, and the result will be nearly bit-identical to what's on the build farm. Hope this helps.
- norswap 12y ago
  Thanks! Would it be possible to have a declarative (mostly, I'm interested about multi-versioning and rollbacks) using the .deb / .rpm binaries instead of the sources?
- SwellJoe 12y ago
  "Nix/Guix are source-based distributions. They build packages in an isolated chroot, in which only the explicit dependencies of a package are available. This means that no user must rely on a server to give them binaries. Any user can opt to build any package on their own system instead, and the result will be nearly bit-identical to what's on the build farm." While the package format RPM lacks insurance of this process, the culture of Fedora/RHEL is that packages are built using mock (as part of the koji build system), which does build everything in a chroot. And, if a package would build differently (such as pulling in different libraries, for instance) for someone using the same tools it is considered a bug and will be fixed by the maintainer. I don't know how specific guix/nix get in this regard: Do they specify the build dependencies down to some sort of file-based checksum level? Or will they build against any compatible version of a library? If the latter, how is that different from RPM when using mock? One could just as readily end up with a very different build of a package, based on which library version is available in the repo against which the package is being built. One could lock down the remote repository to create predictable builds (i.e. only use the Fedora version 20 repo, with no updates; or the Guix version whatever repo with no updates), and presumably either would produce very predictable builds (I know the Fedora case would). So, what do you mean when you say "reproducible"? Is it really specific and unforgiving of new versions of things? Or, does it allow for newer versions easily? RPM allows one to specify things with extreme specificity (down to the exact version and package revision), but one rarely does that if the package will build successfully with a range of versions. I'm not sure I'm convinced this is a unique feature of Guix/Nix, is what I'm trying to say.
  - davexunit 12y ago
    >Do they specify the build dependencies down to some sort of file-based checksum level? Yes, Emacs 24.4 built with X support has a different hash than Emacs 24.4 built without X support. They are completey different in the eyes on Nix/Guix. >So, what do you mean when you say "reproducible"? Is it really specific and unforgiving of new versions of things? Or, does it allow for newer versions easily? In Nix/Guix, you have to specify the exact dependencies, nothing is implicit. Packages are built as a pure function. A package's identity is the hash of all its build inputs: source code, build system, and other dependencies. Bumping the version of a dependency triggers rebuilds of packages that depend on it. I know of no other package management systems that treat package builds in this manner.
    - SwellJoe 12y ago
      This doesn't really sound like a feature. Or, at least, not a feature I've ever wished I had. From a purely pragmatic perspective, it means huge swaths of software have to be rebuilt any time anything gets updated. And, it's become clear now to me that every user is building all their own software (as with ports or Gentoo ebuild, both of which I consider unacceptable package managers for servers for that very reason)...given that, and given that rebuilds now must often span huge piles of software, occasionally with exploding dependency chains, I'm feeling a little queasy imagining trying to administer such a system. I understand the appeal, from a security perspective (and particularly in a world in which the NSA and other highly capable attackers are willing to exert resources to compromise software at the vendor level), of being able to duplicate a package from source. That's cool. But, why force every user to do so every time they update their software? Am I missing something? Are there also binary packages and being able to verify package builds is optional?
      - davexunit 12y ago
        Users do not need to build everything for themselves. They can download binaries from the build farm, just as if they were using a binary-based distro. The system is transparent: If a binary is available from an authorized remote machine, Guix will pull from it. If not, Guix will build it locally. To respond to security updates to core packages a lot faster, we are using a technique called 'grafting' that doesn't involve full rebuilds.
wtetzner 12y ago
Yeah, I think there is metadata missing from those repositories. Nix uniquely identifies a package by hashing all of the inputs that went into building it.
Doji 12y ago
Can you clarify what's wrong with nixpkgs? https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs Granted, it's not yet perfect, but my experience with it so far has left me very impressed.

by norswap 12y ago

davexunit 12y ago

>why don't they use the Debian / Fedora repositories? They are fundamentally incompatible. Among other reasons, Debian and Fedora package builds are not reproducible. That is, given the corresponding source code to the package, you're not likely to get the same binary if you build it yourself. Sometimes maintainer uploaded packages may not build at all for someone else, because the binary built on the maintainer's machine relied on some extra software not specified in the build recipe. Nix/Guix are source-based distributions. They build packages in an isolated chroot, in which only the explicit dependencies of a package are available. This means that no user must rely on a server to give them binaries. Any user can opt to build any package on their own system instead, and the result will be nearly bit-identical to what's on the build farm. Hope this helps.

norswap 12y ago
Thanks! Would it be possible to have a declarative (mostly, I'm interested about multi-versioning and rollbacks) using the .deb / .rpm binaries instead of the sources?
SwellJoe 12y ago
"Nix/Guix are source-based distributions. They build packages in an isolated chroot, in which only the explicit dependencies of a package are available. This means that no user must rely on a server to give them binaries. Any user can opt to build any package on their own system instead, and the result will be nearly bit-identical to what's on the build farm." While the package format RPM lacks insurance of this process, the culture of Fedora/RHEL is that packages are built using mock (as part of the koji build system), which does build everything in a chroot. And, if a package would build differently (such as pulling in different libraries, for instance) for someone using the same tools it is considered a bug and will be fixed by the maintainer. I don't know how specific guix/nix get in this regard: Do they specify the build dependencies down to some sort of file-based checksum level? Or will they build against any compatible version of a library? If the latter, how is that different from RPM when using mock? One could just as readily end up with a very different build of a package, based on which library version is available in the repo against which the package is being built. One could lock down the remote repository to create predictable builds (i.e. only use the Fedora version 20 repo, with no updates; or the Guix version whatever repo with no updates), and presumably either would produce very predictable builds (I know the Fedora case would). So, what do you mean when you say "reproducible"? Is it really specific and unforgiving of new versions of things? Or, does it allow for newer versions easily? RPM allows one to specify things with extreme specificity (down to the exact version and package revision), but one rarely does that if the package will build successfully with a range of versions. I'm not sure I'm convinced this is a unique feature of Guix/Nix, is what I'm trying to say.
- davexunit 12y ago
  >Do they specify the build dependencies down to some sort of file-based checksum level? Yes, Emacs 24.4 built with X support has a different hash than Emacs 24.4 built without X support. They are completey different in the eyes on Nix/Guix. >So, what do you mean when you say "reproducible"? Is it really specific and unforgiving of new versions of things? Or, does it allow for newer versions easily? In Nix/Guix, you have to specify the exact dependencies, nothing is implicit. Packages are built as a pure function. A package's identity is the hash of all its build inputs: source code, build system, and other dependencies. Bumping the version of a dependency triggers rebuilds of packages that depend on it. I know of no other package management systems that treat package builds in this manner.
  - SwellJoe 12y ago
    This doesn't really sound like a feature. Or, at least, not a feature I've ever wished I had. From a purely pragmatic perspective, it means huge swaths of software have to be rebuilt any time anything gets updated. And, it's become clear now to me that every user is building all their own software (as with ports or Gentoo ebuild, both of which I consider unacceptable package managers for servers for that very reason)...given that, and given that rebuilds now must often span huge piles of software, occasionally with exploding dependency chains, I'm feeling a little queasy imagining trying to administer such a system. I understand the appeal, from a security perspective (and particularly in a world in which the NSA and other highly capable attackers are willing to exert resources to compromise software at the vendor level), of being able to duplicate a package from source. That's cool. But, why force every user to do so every time they update their software? Am I missing something? Are there also binary packages and being able to verify package builds is optional?
    - davexunit 12y ago
      Users do not need to build everything for themselves. They can download binaries from the build farm, just as if they were using a binary-based distro. The system is transparent: If a binary is available from an authorized remote machine, Guix will pull from it. If not, Guix will build it locally. To respond to security updates to core packages a lot faster, we are using a technique called 'grafting' that doesn't involve full rebuilds.

wtetzner 12y ago

Yeah, I think there is metadata missing from those repositories. Nix uniquely identifies a package by hashing all of the inputs that went into building it.

Doji 12y ago

Can you clarify what's wrong with nixpkgs? https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs Granted, it's not yet perfect, but my experience with it so far has left me very impressed.