5 ms·

2) In Debian a binary package has almost always just been a cache, are we thinking about Debian packaging in completely different ways? Are you perhaps talking

2) In Debian a binary package has almost always just been a cache, are we thinking about Debian packaging in completely different ways? Are you perhaps talking about the aspect of doing an "build world"[0] that bsd, Gentoo and now Nix are better at? [0] Edit: rebuilding your dependencies if you need it and handling that seemlesly CAN be hard on Debian. Something that even Nix struggles with even if they are best in class by far. It is also completely different from the notion of compiling a program. I wonder what you consider the goal of a package system.

tazjin 4y ago
> are we thinking about Debian packaging in completely different ways? Quite likely! The whole concept of separately building a source/binary package, and then uploading/"deploying" that binary package, already violates the notion of being "just a cache" for me. There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix (i.e. guarantees about the artifact being repeatable, and being addressable by its inputs rather than a human-curated tag or version number). > Something that even Nix struggles with Nix only struggles with it to the degree that some of the higher-level abstractions in the existing package set (which are built on top of the fundamental model, not part of it) can be confusing/underdocumented, but conceptually this is a simple thing to do in Nix. Even practically at this point it is usually simple - unless you're dealing with Haskell or Javascript, of course. > I wonder what you consider the goal of a package system I want it to let me describe a desired state, and then make it so. That state is best represented as a graph, like a Merkle tree, of the instructions for the individual steps and a way to address them. An individual step is a transformation (i.e. some program) executed over some sources, yielding some result (likely in the filesystem). I want any distribution of binaries to be a result of using this addressing scheme, and looking up/copying an already built equivalent artifact for that thing. I want a text file containing the string "Hello HN" to be represented by the same abstraction that represents a specific .so file, the `emacs` package, the configuration of my entire system, the configuration of a cluster of systems and so on. I want this system to be programmable so I can work around the shortcomings that its original designers missed. Nix (and Guix) do large parts of this already, and are conceptually (though not yet technically) capable of doing all of it. Technically, so is something like Bazel - but its complexity and maintenance requirements make that prohibitively expensive (basically only Google can use Bazel like that, and even they have delineated areas where they "give up" on wrapping things in Bazel).
- Nursie 4y ago
  > There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" apt-get --build source package_name It's unlikely to be fully repeatable to the point of producing an identical binary, but it's also not some esoteric set of procedures.
- goodpoint 4y ago
  > There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.
  - anderskaseorg 4y ago
    Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.html https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU. In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility. Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.
    - 0x0 4y ago
      Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.
pjc50 4y ago
Debian packages can have pre/postinstall scripts that mutate the overall state of the system in arbitrary ways. This is both (a) necessary to make some thing work and (b) ruins the theoretical model.
- Nursie 4y ago
  Those dirty practicalities breaking my perfect mental model again! (This is a huge issue in our field in all sorts of ways, and a constant source of both pain and compromise)
  - pxc 4y ago
    Alternatively: that pesky lack of discipline obstructing my optimizations and features again.
- marcosdumay 4y ago
  > This is ... necessary to make some thing work Things work in NixOS without the need of any of that. So, no it is not necessary. It's a need created purely from the other directives of the Debian organization.
  - hiljusti 4y ago
    Not necessary, but certainly it's easier to implement and get stuff done. It starts to break down and show its cracks the larger and larger it gets, though. The difference between almost all Linux or BSD packaging systems and nixpkgs is like the difference between dynamic and compile-time-type-checked programming languages.
    - pxc 4y ago
      You can have a much more conventional package manager without pre- or post-install hooks. One ex-Debian developer created such a package manager as a research project, presumably to prove a point to other Debian people with the same head-in-the-sand attitude about the state of Debian's package tooling. It doesn't require the same kind of patching as Nix does (although sometimes patching does help avoid requiring those scripts). Check it out: https://distr1.org/ https://distr1.org/

tazjin 4y ago

> are we thinking about Debian packaging in completely different ways? Quite likely! The whole concept of separately building a source/binary package, and then uploading/"deploying" that binary package, already violates the notion of being "just a cache" for me. There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix (i.e. guarantees about the artifact being repeatable, and being addressable by its inputs rather than a human-curated tag or version number). > Something that even Nix struggles with Nix only struggles with it to the degree that some of the higher-level abstractions in the existing package set (which are built on top of the fundamental model, not part of it) can be confusing/underdocumented, but conceptually this is a simple thing to do in Nix. Even practically at this point it is usually simple - unless you're dealing with Haskell or Javascript, of course. > I wonder what you consider the goal of a package system I want it to let me describe a desired state, and then make it so. That state is best represented as a graph, like a Merkle tree, of the instructions for the individual steps and a way to address them. An individual step is a transformation (i.e. some program) executed over some sources, yielding some result (likely in the filesystem). I want any distribution of binaries to be a result of using this addressing scheme, and looking up/copying an already built equivalent artifact for that thing. I want a text file containing the string "Hello HN" to be represented by the same abstraction that represents a specific .so file, the `emacs` package, the configuration of my entire system, the configuration of a cluster of systems and so on. I want this system to be programmable so I can work around the shortcomings that its original designers missed. Nix (and Guix) do large parts of this already, and are conceptually (though not yet technically) capable of doing all of it. Technically, so is something like Bazel - but its complexity and maintenance requirements make that prohibitively expensive (basically only Google can use Bazel like that, and even they have delineated areas where they "give up" on wrapping things in Bazel).

Nursie 4y ago
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" apt-get --build source package_name It's unlikely to be fully repeatable to the point of producing an identical binary, but it's also not some esoteric set of procedures.
goodpoint 4y ago
> There might be a tool in Debian where I can seamlessly say "do not download this package from the repository, but build it locally" - but even if so, it would surprise me if it can give me the same guarantees as Nix It's been there for 15 or 20 years. Also Debian has been doing reproducible builds before Nix.
- anderskaseorg 4y ago
  Even in the best case where Debian satisfies all the reproducibility goals it sets for itself (i.e. the ~96% at https://tests.reproducible-builds.org/debian/reproducible.html https://tests.reproducible-builds.org/debian/reproducible.ht... becomes 100%), each Debian package is still only a reproducible function of the current versions of its build dependencies at the time it was built. Upgrading those dependencies can change the build output, and this is still normal and expected in Debian; it’s the entire point of a binNMU. In order to reproduce a given Debian package from scratch, one would need to track down long chains of historical versions of packages from sid. Due to circular dependencies, these chains probably snake back to the very first versions of Debian before the availability of snapshot.debian.org. It’s possible that human judgment could be used to break some chains at an earlier point by substituting different versions, but this goes outside the guarantees provided by Debian’s reproducibility. Nix guarantees that every package is built with the specified versions of its dependencies, and so on recursively. Some more rebuilding is necessary to maintain this guarantee, but in exchange, it enables stronger notions of reproducibility that Debian can’t.
  - 0x0 4y ago
    Well of course it snakes back, and likely beyond debian's inception, your initial c compiler has to come from somewhere.

pjc50 4y ago

Debian packages can have pre/postinstall scripts that mutate the overall state of the system in arbitrary ways. This is both (a) necessary to make some thing work and (b) ruins the theoretical model.

Nursie 4y ago
Those dirty practicalities breaking my perfect mental model again! (This is a huge issue in our field in all sorts of ways, and a constant source of both pain and compromise)
- pxc 4y ago
  Alternatively: that pesky lack of discipline obstructing my optimizations and features again.
marcosdumay 4y ago
> This is ... necessary to make some thing work Things work in NixOS without the need of any of that. So, no it is not necessary. It's a need created purely from the other directives of the Debian organization.
- hiljusti 4y ago
  Not necessary, but certainly it's easier to implement and get stuff done. It starts to break down and show its cracks the larger and larger it gets, though. The difference between almost all Linux or BSD packaging systems and nixpkgs is like the difference between dynamic and compile-time-type-checked programming languages.
  - pxc 4y ago
    You can have a much more conventional package manager without pre- or post-install hooks. One ex-Debian developer created such a package manager as a research project, presumably to prove a point to other Debian people with the same head-in-the-sand attitude about the state of Debian's package tooling. It doesn't require the same kind of patching as Nix does (although sometimes patching does help avoid requiring those scripts). Check it out: https://distr1.org/ https://distr1.org/