3 ms·

> are we thinking about Debian packaging in completely different ways? Quite likely! The whole concept of separately building a source/binary package, and then

by tazjin 4y ago

> are we thinking about Debian packaging in completely different ways? Quite likely! The whole concept of separately building a source/binary package, and then uploading/"deploying" that binary package, already violates the notion of being "just a cache" for me. There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix (i.e. guarantees about the artifact being repeatable, and being addressable by its inputs rather than a human-curated tag or version number). > Something that even Nix struggles with Nix only struggles with it to the degree that some of the higher-level abstractions in the existing package set (which are built on top of the fundamental model, not part of it) can be confusing/underdocumented, but conceptually this is a simple thing to do in Nix. Even practically at this point it is usually simple - unless you're dealing with Haskell or Javascript, of course. > I wonder what you consider the goal of a package system I want it to let me describe a desired state, and then make it so. That state is best represented as a graph, like a Merkle tree, of the instructions for the individual steps and a way to address them. An individual step is a transformation (i.e. some program) executed over some sources, yielding some result (likely in the filesystem). I want any distribution of binaries to be a result of using this addressing scheme, and looking up/copying an already built equivalent artifact for that thing. I want a text file containing the string "Hello HN" to be represented by the same abstraction that represents a specific .so file, the `emacs` package, the configuration of my entire system, the configuration of a cluster of systems and so on. I want this system to be programmable so I can work around the shortcomings that its original designers missed. Nix (and Guix) do large parts of this already, and are conceptually (though not yet technically) capable of doing all of it. Technically, so is something like Bazel - but its complexity and maintenance requirements make that prohibitively expensive (basically only Google can use Bazel like that, and even they have delineated areas where they "give up" on wrapping things in Bazel).

Nursie 4y ago
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" apt-get --build source package_name It's unlikely to be fully repeatable to the point of producing an identical binary, but it's also not some esoteric set of procedures.
goodpoint 4y ago
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.
- anderskaseorg 4y ago
  Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.html https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU. In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility. Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.
  - 0x0 4y ago
    Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.
    - anderskaseorg 4y ago
      Right. Nix bootstraps from a fixed collection of seed binaries, and everything in any given commit of nixpkgs is derived from the seed binaries and versions of software that exist in that commit of nixpkgs. It doesn’t solve the “trusting trust” problem with those seed binaries, but it does solve the “arbitrarily long chains” problem. (The work at https://www.gnu.org/software/mes/ https://www.gnu.org/software/mes/ aims to minimize the needed binary seed to a human-auditable size, and should be applicable to Nix and Guix.)

by tazjin 4y ago

Nursie 4y ago

> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" apt-get --build source package_name It's unlikely to be fully repeatable to the point of producing an identical binary, but it's also not some esoteric set of procedures.

goodpoint 4y ago

> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.

anderskaseorg 4y ago
Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.html https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU. In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility. Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.
- 0x0 4y ago
  Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.
  - anderskaseorg 4y ago
    Right. Nix bootstraps from a fixed collection of seed binaries, and everything in any given commit of nixpkgs is derived from the seed binaries and versions of software that exist in that commit of nixpkgs. It doesn’t solve the “trusting trust” problem with those seed binaries, but it does solve the “arbitrarily long chains” problem. (The work at https://www.gnu.org/software/mes/ https://www.gnu.org/software/mes/ aims to minimize the needed binary seed to a human-auditable size, and should be applicable to Nix and Guix.)