7 ms·

Reproducible Arch Linux

243 points by nwah1 9y ago

apetresc 9y ago
Headline is misleading – according to the article, 76% of the 17% of packages that are being tested are reproducible.
- hexane360 9y ago
  If it's a random sample, that seems to be a reasonable extrapolation (of course standard caveats apply).
- Foxboron 9y ago
  The blog is missing the link of the page. The awesome folks over at debian, mainly Holger Levsen, is helping Arch Linux achieve this with their infrastructure. The progress and status can be seen here: https://tests.reproducible-builds.org/archlinux/archlinux.html https://tests.reproducible-builds.org/archlinux/archlinux.ht... Currently at 76% of 70% built packages.
  - Palomides 9y ago
    for comparison, debian stretch on amd64 is at 94.1% of 24821 packages. glad to see the knowledge is being shared. stats: https://tests.reproducible-builds.org/debian/reproducible.html https://tests.reproducible-builds.org/debian/reproducible.ht...
    - jelly1 9y ago
      We work together with a few Debian developers involved with reproducible builds in the reproducible-builds project, so yes a lot of work is shared which is awesome :)
- vorotato 9y ago
  http://a.co/a8yEzzt http://a.co/a8yEzzt A Manga Guide to Statistics. No I'm not being snarky, it's a great read and I highly recommend it. Sampling, why it works and the philosophy behind statistics are all a bit unintuitive and so a light read is perfect for helping you develop that intuition.
  - hnarn 9y ago
    >A Manga Guide No thanks.
  - apetresc 9y ago
    I have a degree in mathematics, I'm fine on the statistics front. It's very likely to NOT be a random sample - you would only be testing the packages which have had effort put in to make them reproducible, for reproducibility. I would expect the non-sampled packages to be 0% reproducible, since the reason they were not tested is that nobody is trying to make them reproducible (yet).
- jelly1 9y ago
  Well I didn't mean to claim it's 100% reproducible in the title. But we finally started working towards reproducible builds, we still need a way for users to easily reproduce builds, publish the results somewhere. And test with more differences, luckily Debian has done a lot of effort in this regard!
magpi3 9y ago
For those like myself who have never heard of the term: https://reproducible-builds.org https://reproducible-builds.org
kmicklas 9y ago
Arch with reproducible builds is called NixOS. No but in all seriousness, maybe there should be some unification of efforts. The distros share a similar philosophy modulo Nix's better package manager and Arch's better packaging.
djsumdog 9y ago
I have not heard of this concept for binary distributions. It's a great idea. I've been living in Gentoo world for over a decade, so of course, every Gentoo system is going to be a little (or a lot) different. :-P
sowbug 9y ago
Tails is a smaller, specialized distribution that recently reached this milestone: https://tails.boum.org/news/reproducible_Tails/index.en.html https://tails.boum.org/news/reproducible_Tails/index.en.html Given the goal of Tails to provide a private computing environment that leaves no traces on the host system after shutdown, it's especially valuable to be able to confirm that the source truly produces the same binary image that the site offers for download. (If anyone from Tails is reading, please update libccid to 1.4.27!)
- jelly1 9y ago
  In the future, Arch also wants to make their iso also reproducible :-). But first the packages have to be reproducible.
rdtsc 9y ago
How does this relate to what NixOS is doing? I've been casually following it. I know they build everything from source and then build results get hashed. The hash is not just the output, but also contains all input of the build. The assumption is that once a package and its dependencies are built, it will end up producing the same output. Is the difference then that reproducible Arch or say Debian would ensure that the source when compiled would produce the same binary output on _any_ the machine of the same type. As opposed to say maybe NixOS would just make the output different if it runs on a machine with a hard drive mounted on a different directory or the cpu has more cores?
- aseipp 9y ago
  You're close. For NixOS, I distinguish between "determinism" and "reproducible": - Deterministic: the same input always gives the same output source, no exceptions. - Reproducible: The steps to *create* some build result are always the same, but the results might not be bit-for-bit identical. (Please note that for most people they use "Reproducible" to refer to what I call "Determinism", but this is only in the context of NixOS, so just go with me.) NixOS is always reproducible. Any build can be exactly reproduced by another person, and the build will run the exact same steps. But it is not always deterministic. This seems strange but it really isn't. For example, although I may always take the same steps to compile some software (let's say "./configure; make; make install"), that does not guarantee determinism -- for example, perhaps the 'make' stage will run `date` and then embed the date in the source (e.g. `gcc -DBUILD_DATE="$(date)"` or something). That breaks determinism. However, Nix is always reproducible. If you run 'nix-build' today, tomorrow, or 5 years from now -- if you have the same description of the package, the build always proceeds the same way. In practice, this means "You need to have the same git revision of the nixpkgs source code". But if you have that -- nothing should ever change. Of course you can monkey-wrench this and make up various weird counter examples to this ("My build system only compiles stuff when rand() % 4 == 10, hah!"), but that's the general idea. NixOS will always take the exact same steps -- down to the letter -- in order to build your software. But that doesn't guarantee its deterministic. Most other systems are, in a sense, also reproducible, because you could just "run all the commands the same way, every time" -- but not in the highly automated way Nix is. In general a single hash identifies everything, to the point that I can literally recreate exact copies of a machine with a single command. I can reformat my laptop and have it back to the exact same software in 10 minutes, etc. It is also more nuanced than that: on something like Arch Linux, you and I may be running two Arch machines. They may feel identical. We might even both run './configure; make; make install' and the software will work similarly. But if I haven't run 'pacman -Syu' in week, and you ran it yesterday -- that isn't the same thing! Our environments are not reproductions of each other; they are only vaguely similar. Maybe you got a bugfixed glibc for example that I did not. If that was the case in NixOS, it would cause a rebuild (because one of the inputs -- some of the steps you must take to create something, including installing its prerequisites -- have changed). In this case, "reproducible" does not mean "this piece of software is obtained by running this exact set of steps", that's too narrow. It means "this result is obtained by running the exact set of steps, for every single thing this result has ever depended on, all the way up the chain". > As opposed to say maybe NixOS would just make the output different if it runs on a machine with a hard drive mounted on a different directory or the cpu has more cores? No, this won't happen. CPU cores and the "location" of the source code are not included in the hash[1], they are not part of the input "source" to a build result. [1] What actually happens is that the source code is copied into a location inside the "Nix store", and it is built from there. Same source code? Same destination. So in other words the first thing Nix really does is "create an build result from the input source code", i.e. a directory containing a copy of all the code, and that is given as the input to another step that actually builds the results, using the source as input. You could think of "building software" as two packages: one package, which is built by unzipping the source code and keeping the results, and another that is created by building the previously unzipped code. So it uses the same mechanism as it always does: source code is just an input to something we want to "build", in this case, the executables. It's no different than "zlib" the library an input to "libpng" the library (because zlib needs libpng). It is just an "input" in an abstract sense.
xaduha 9y ago
The more the merrier. I'll be sticking to NixOS in the meantime.
- XR0CSWV3h3kZWg 9y ago
  So far I have been experimenting with using nix package management on top of other OSes, how much extra do you feel like running NixOS adds? I've certainly run into missing packages a fair bit, so far nothing beats the coverage of the AUR.
  - xaduha 9y ago
    I'm mostly using it as a base, as a hypervisor of sorts for docker and libvirt. Anything else I need I can access with those. State of the system declaratively described in a few files, what's not to like? From limited experience I had with AUR - for not so rigorously maintained packages it's not much better than building from the source yourself.
    - XR0CSWV3h3kZWg 9y ago
      My main issue so far has been that the spec language isn't as simple as I hoped it would be and there are many packages missing or not up to date. The ability to try out a new package and safely downgrade has been exactly as it says on the tin. So far I have primarily used it as an add on to other OSes and put it in the PATH before everything else. That way if something is missing in the nix packages I'll fall back to the other OS. And yes the AUR has no standard of quality, but the coverage is greater than anything I have ever seen. Figuring out exactly what build system and assumptions a package might have made is something I only do once the AUR package has failed.

243 points by nwah1 9y ago

apetresc 9y ago

Headline is misleading – according to the article, 76% of the 17% of packages that are being tested are reproducible.

hexane360 9y ago
If it's a random sample, that seems to be a reasonable extrapolation (of course standard caveats apply).
Foxboron 9y ago
The blog is missing the link of the page. The awesome folks over at debian, mainly Holger Levsen, is helping Arch Linux achieve this with their infrastructure. The progress and status can be seen here: https://tests.reproducible-builds.org/archlinux/archlinux.html https://tests.reproducible-builds.org/archlinux/archlinux.ht... Currently at 76% of 70% built packages.
- Palomides 9y ago
  for comparison, debian stretch on amd64 is at 94.1% of 24821 packages. glad to see the knowledge is being shared. stats: https://tests.reproducible-builds.org/debian/reproducible.html https://tests.reproducible-builds.org/debian/reproducible.ht...
  - jelly1 9y ago
    We work together with a few Debian developers involved with reproducible builds in the reproducible-builds project, so yes a lot of work is shared which is awesome :)
vorotato 9y ago
http://a.co/a8yEzzt http://a.co/a8yEzzt A Manga Guide to Statistics. No I'm not being snarky, it's a great read and I highly recommend it. Sampling, why it works and the philosophy behind statistics are all a bit unintuitive and so a light read is perfect for helping you develop that intuition.
- hnarn 9y ago
  >A Manga Guide No thanks.
- apetresc 9y ago
  I have a degree in mathematics, I'm fine on the statistics front. It's very likely to NOT be a random sample - you would only be testing the packages which have had effort put in to make them reproducible, for reproducibility. I would expect the non-sampled packages to be 0% reproducible, since the reason they were not tested is that nobody is trying to make them reproducible (yet).
jelly1 9y ago
Well I didn't mean to claim it's 100% reproducible in the title. But we finally started working towards reproducible builds, we still need a way for users to easily reproduce builds, publish the results somewhere. And test with more differences, luckily Debian has done a lot of effort in this regard!

magpi3 9y ago

For those like myself who have never heard of the term: https://reproducible-builds.org https://reproducible-builds.org

kmicklas 9y ago

Arch with reproducible builds is called NixOS. No but in all seriousness, maybe there should be some unification of efforts. The distros share a similar philosophy modulo Nix's better package manager and Arch's better packaging.

djsumdog 9y ago

I have not heard of this concept for binary distributions. It's a great idea. I've been living in Gentoo world for over a decade, so of course, every Gentoo system is going to be a little (or a lot) different. :-P

sowbug 9y ago

Tails is a smaller, specialized distribution that recently reached this milestone: https://tails.boum.org/news/reproducible_Tails/index.en.html https://tails.boum.org/news/reproducible_Tails/index.en.html Given the goal of Tails to provide a private computing environment that leaves no traces on the host system after shutdown, it's especially valuable to be able to confirm that the source truly produces the same binary image that the site offers for download. (If anyone from Tails is reading, please update libccid to 1.4.27!)

jelly1 9y ago
In the future, Arch also wants to make their iso also reproducible :-). But first the packages have to be reproducible.

rdtsc 9y ago

How does this relate to what NixOS is doing? I've been casually following it. I know they build everything from source and then build results get hashed. The hash is not just the output, but also contains all input of the build. The assumption is that once a package and its dependencies are built, it will end up producing the same output. Is the difference then that reproducible Arch or say Debian would ensure that the source when compiled would produce the same binary output on _any_ the machine of the same type. As opposed to say maybe NixOS would just make the output different if it runs on a machine with a hard drive mounted on a different directory or the cpu has more cores?

aseipp 9y ago
You're close. For NixOS, I distinguish between "determinism" and "reproducible": - Deterministic: the same input always gives the same output source, no exceptions. - Reproducible: The steps to *create* some build result are always the same, but the results might not be bit-for-bit identical. (Please note that for most people they use "Reproducible" to refer to what I call "Determinism", but this is only in the context of NixOS, so just go with me.) NixOS is always reproducible. Any build can be exactly reproduced by another person, and the build will run the exact same steps. But it is not always deterministic. This seems strange but it really isn't. For example, although I may always take the same steps to compile some software (let's say "./configure; make; make install"), that does not guarantee determinism -- for example, perhaps the 'make' stage will run `date` and then embed the date in the source (e.g. `gcc -DBUILD_DATE="$(date)"` or something). That breaks determinism. However, Nix is always reproducible. If you run 'nix-build' today, tomorrow, or 5 years from now -- if you have the same description of the package, the build always proceeds the same way. In practice, this means "You need to have the same git revision of the nixpkgs source code". But if you have that -- nothing should ever change. Of course you can monkey-wrench this and make up various weird counter examples to this ("My build system only compiles stuff when rand() % 4 == 10, hah!"), but that's the general idea. NixOS will always take the exact same steps -- down to the letter -- in order to build your software. But that doesn't guarantee its deterministic. Most other systems are, in a sense, also reproducible, because you could just "run all the commands the same way, every time" -- but not in the highly automated way Nix is. In general a single hash identifies everything, to the point that I can literally recreate exact copies of a machine with a single command. I can reformat my laptop and have it back to the exact same software in 10 minutes, etc. It is also more nuanced than that: on something like Arch Linux, you and I may be running two Arch machines. They may feel identical. We might even both run './configure; make; make install' and the software will work similarly. But if I haven't run 'pacman -Syu' in week, and you ran it yesterday -- that isn't the same thing! Our environments are not reproductions of each other; they are only vaguely similar. Maybe you got a bugfixed glibc for example that I did not. If that was the case in NixOS, it would cause a rebuild (because one of the inputs -- some of the steps you must take to create something, including installing its prerequisites -- have changed). In this case, "reproducible" does not mean "this piece of software is obtained by running this exact set of steps", that's too narrow. It means "this result is obtained by running the exact set of steps, for every single thing this result has ever depended on, all the way up the chain". > As opposed to say maybe NixOS would just make the output different if it runs on a machine with a hard drive mounted on a different directory or the cpu has more cores? No, this won't happen. CPU cores and the "location" of the source code are not included in the hash[1], they are not part of the input "source" to a build result. [1] What actually happens is that the source code is copied into a location inside the "Nix store", and it is built from there. Same source code? Same destination. So in other words the first thing Nix really does is "create an build result from the input source code", i.e. a directory containing a copy of all the code, and that is given as the input to another step that actually builds the results, using the source as input. You could think of "building software" as two packages: one package, which is built by unzipping the source code and keeping the results, and another that is created by building the previously unzipped code. So it uses the same mechanism as it always does: source code is just an input to something we want to "build", in this case, the executables. It's no different than "zlib" the library an input to "libpng" the library (because zlib needs libpng). It is just an "input" in an abstract sense.

xaduha 9y ago

The more the merrier. I'll be sticking to NixOS in the meantime.

XR0CSWV3h3kZWg 9y ago
So far I have been experimenting with using nix package management on top of other OSes, how much extra do you feel like running NixOS adds? I've certainly run into missing packages a fair bit, so far nothing beats the coverage of the AUR.
- xaduha 9y ago
  I'm mostly using it as a base, as a hypervisor of sorts for docker and libvirt. Anything else I need I can access with those. State of the system declaratively described in a few files, what's not to like? From limited experience I had with AUR - for not so rigorously maintained packages it's not much better than building from the source yourself.
  - XR0CSWV3h3kZWg 9y ago
    My main issue so far has been that the spec language isn't as simple as I hoped it would be and there are many packages missing or not up to date. The ability to try out a new package and safely downgrade has been exactly as it says on the tin. So far I have primarily used it as an add on to other OSes and put it in the PATH before everything else. That way if something is missing in the nix packages I'll fall back to the other OS. And yes the AUR has no standard of quality, but the coverage is greater than anything I have ever seen. Figuring out exactly what build system and assumptions a package might have made is something I only do once the AUR package has failed.