3 ms·

But how do you reproduce the dpkg packages? When your friend hands you a deb, instead of a docker image, what have you accomplished as far as knowing how they a

But how do you reproduce the dpkg packages? When your friend hands you a deb, instead of a docker image, what have you accomplished as far as knowing how they arrived at their configuration? Debs are also binary blobs, just more of them, and separately. From what I can see, there are two philosophically pure approaches to software packaging. Either we can build everything from source, so that we have reproducibility (see e.g. emerge for a practical system, although it is still not bit-for-bit reproducible) or else we can ship everything as a blob so we have total consistency (e.g. Docker, Go). The other approaches, apt and nix, seem to me to be compromise positions. They may even be practical or useful compromises. But they are ultimately about the degree to which we allow binary blobs, not whether we do. But it seems to me that once you have accepted installing binaries via apt then installing them from Docker is a question of degree, not of kind. It seems very hard to me to argue that distributing binaries via Docker is really worse than via nix.

colemickens 10y ago
Hm. Can you elaborate on why nix is a compromise? Nix is effectively building from source, but with lots of caching in place to avoid end users constantly having to rebuild every last package from source. Nix and Apt are very different in this regard. >But they are ultimately about the degree to which we allow binary blobs, not whether we do. I don't agree with this assesment of Nix and Nix packages at all. My local system evaluates the package definition (a description of how to build it from source) and merely is smart enough to retrieve a prebuilt binary with the exact same source inputs and configuration. Sure, I'm still trusting the remote party to have built it as described, but that's true of any system where pre-built binaries are distributed. For another explanation, I could disable the NixOS Hydra Cache and could theoretically rebuild my current system state without their binary cache and still wind up with the exact same bits. At least, that's the idea behind Nix.
pmahoney 10y ago
> Either we can build everything from source, so that we have reproducibility (see e.g. emerge for a practical system, although it is still not bit-for-bit reproducible) or else we can ship everything as a blob so we have total consistency (e.g. Docker, Go). Nix is really both of these combined, and is not like apt/dpkg in how it separates source and binary packages. A Nix package is really instructions (including all compiler flags) of how to build something from source [1] and nothing more. Due to being pure functions, given the binary inputs (e.g. a zlib dependency previously built) and the instructions, any two people will always get the same binary output upon executing those instructions [2]. Furthermore, without executing the instructions, I can generate the unique identifier of the resulting binary package by hashing the inputs and the instructions. This unique identifier can be used to determine if the package already exists locally in /nix/store, and optionally be used to check if some trusted binary cache (such as Nix's binary cache) already has the binary package. There is no binary package in the sense that apt/dpkg have source packages and many derived binary packages which are installed without need of the source package. For example (contrived and simplified): gcc-uniqueGccHash = bootstrappingProblem() libc-uniqueLibcHash = buildLibc(gcc-uniqueGccHash) zlib-uniqueZlibHash = buildZlib(gcc..., libc...) myprog-uniqueMyProgHash = buildMyProg(gcc..., libc..., zlib...) To download "myprog" from a binary cache, I can't calculate uniqueMyProgHash using any old versions of libc and zlib; I must have exactly the same hashes that were used by the system that built "myprog", all the way up the dependency chain. And I'll only ever have that if the builds are reproducible and/or I downloaded them all from the same binary cache. [1] In practice, there's nothing stopping the instructions from saying "download this binary blob" instead of "download this source tarball". [2] The problem of build reproduciblity is not solved, but many outside of Nix also see this as important, see https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds
koverstreet 10y ago
It is a major problem - that's why debian has been on a major push for reproducible builds too.

colemickens 10y ago

Hm. Can you elaborate on why nix is a compromise? Nix is effectively building from source, but with lots of caching in place to avoid end users constantly having to rebuild every last package from source. Nix and Apt are very different in this regard. >But they are ultimately about the degree to which we allow binary blobs, not whether we do. I don't agree with this assesment of Nix and Nix packages at all. My local system evaluates the package definition (a description of how to build it from source) and merely is smart enough to retrieve a prebuilt binary with the exact same source inputs and configuration. Sure, I'm still trusting the remote party to have built it as described, but that's true of any system where pre-built binaries are distributed. For another explanation, I could disable the NixOS Hydra Cache and could theoretically rebuild my current system state without their binary cache and still wind up with the exact same bits. At least, that's the idea behind Nix.

pmahoney 10y ago

> Either we can build everything from source, so that we have reproducibility (see e.g. emerge for a practical system, although it is still not bit-for-bit reproducible) or else we can ship everything as a blob so we have total consistency (e.g. Docker, Go). Nix is really both of these combined, and is not like apt/dpkg in how it separates source and binary packages. A Nix package is really instructions (including all compiler flags) of how to build something from source [1] and nothing more. Due to being pure functions, given the binary inputs (e.g. a zlib dependency previously built) and the instructions, any two people will always get the same binary output upon executing those instructions [2]. Furthermore, without executing the instructions, I can generate the unique identifier of the resulting binary package by hashing the inputs and the instructions. This unique identifier can be used to determine if the package already exists locally in /nix/store, and optionally be used to check if some trusted binary cache (such as Nix's binary cache) already has the binary package. There is no binary package in the sense that apt/dpkg have source packages and many derived binary packages which are installed without need of the source package. For example (contrived and simplified): gcc-uniqueGccHash = bootstrappingProblem() libc-uniqueLibcHash = buildLibc(gcc-uniqueGccHash) zlib-uniqueZlibHash = buildZlib(gcc..., libc...) myprog-uniqueMyProgHash = buildMyProg(gcc..., libc..., zlib...) To download "myprog" from a binary cache, I can't calculate uniqueMyProgHash using any old versions of libc and zlib; I must have exactly the same hashes that were used by the system that built "myprog", all the way up the dependency chain. And I'll only ever have that if the builds are reproducible and/or I downloaded them all from the same binary cache. [1] In practice, there's nothing stopping the instructions from saying "download this binary blob" instead of "download this source tarball". [2] The problem of build reproduciblity is not solved, but many outside of Nix also see this as important, see https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds

koverstreet 10y ago

It is a major problem - that's why debian has been on a major push for reproducible builds too.