4 ms·

Just my two cents: Nix doesn’t guarantee “reproducible”, just “repeatable”. (I think it’s helpful to keep these technical concepts distinct, and to introduce th

by jt2190 4y ago

Just my two cents: Nix doesn’t guarantee “reproducible”, just “repeatable”. (I think it’s helpful to keep these technical concepts distinct, and to introduce the right terminology when teaching, since it makes elaboration easier in the future.)

kissgyorgy 4y ago
Can you elaborate? From what I understand, Nix packages are mostly reproducible as they are running great lengths for that, to the point where every time in Nix land is the exact same.
- Filligree 4y ago
  Not bit-for-bit reproducible. Package builds can be nondeterministic, binaries can have embedded timestamps (e.g, build time), and so on. They should be semantically equivalent, but it's not fully guaranteed.
  - dd_ 4y ago
    You're not wrong overall, but your example is. In the build environment, the current time is set to the epoch which removes that variability. An upcoming feature of nix is content addressed builds which use the output hashes to determine the store path. This should make it much more transparent which builds are reproducible and which are not.
    - Filligree 4y ago
      > In the build environment, the current time is set to the epoch which removes that variability. The amount of time per build can vary. Embedded timestamps will all be in January 1970, but even a single second difference means the binaries are no longer bit for bit identical -- and that assumes the embedded timestamp isn't microseconds.
      - infogulch 4y ago
        I think you're assuming that the timestamp only starts at the epoch and still advances in real time, which is not necessarily true.
        Filligree 4y ago
        If it didn't advance, that would cause trouble for some build systems. So yes, I'm assuming it does.
- linuxhackerman 4y ago
  "reproducible" means you get the same result, bit-for-bit, when running a build in different circumstances. It's trivial to write non-reproducible Nix derivations by placing timestamps, randomly generated data, or similar into the build output, directly or indirectly. Nix is a powerful tool for working towards reproducibility, since it can specify the software side of a build environment more exhaustively than most other package managers out there, and has a language (derivations) that enables repeating builds easily as well as comparing them, but does not guarantee reproducibility on its own. A lot of the reproducibility of nixpkgs is attributable to reproducible build efforts elsewhere, including conventions like SOURCE_DATE_EPOCH. Nix itself does not in fact "freeze the clock" for builds, but many tools that use timestamps as part of a build will respect SOURCE_DATE_EPOCH instead of using the system clock. Many of these valuable developments in this direction are coordinated in the cross-distro https://reproducible-builds.org https://reproducible-builds.org group.
- dlahoda 4y ago
  hash of build output is not used in hash of derivation and in derivation tree. hence it is not reproducible.
JamesSwift 4y ago
With flakes it is reproducible in a self-contained way.
baby 4y ago
These two words have the same meaning
- lalopalota 4y ago
  repeatable - do the exact same thing every time reproducable - get the exact same results every time not the same thing
- pxc 4y ago
  Repeating a non-deterministic process can give you a different result, which makes the results non-reproducible. Nix does provide a higher degree of functional reproducibility (builds will have the same behavior, if not identical bits) than other systems do, though, because of the way it locks down exact dependency versions and build flags, though. In that sense, Nix is 'more reproducible' than competing tools even when both are invoking non-deterministic compilers at the last step.
pushtheenvelope 4y ago
would you mind elaborating on the difference?
0cf8612b2e1e 4y ago
Accurate, but there is only so much to be done on that front when the consumed packages do not strive for reproducibility. As Nix or similar tools bring the lack of bit-for-bit reproducibility to the forefront, I hope build systems start adopting processes that will correct this behavior.
tripdout 4y ago
I mean, its impossible to guarantee full reproducibility under all possible conditions, but Nix does take steps towards promoting it. eg setting all times to UNIX epoch, enforcing purity (so specifying dependencies explicitly instead of fetching them from $PATH, etc).

by jt2190 4y ago

kissgyorgy 4y ago

Can you elaborate? From what I understand, Nix packages are mostly reproducible as they are running great lengths for that, to the point where every time in Nix land is the exact same.

Filligree 4y ago
Not bit-for-bit reproducible. Package builds can be nondeterministic, binaries can have embedded timestamps (e.g, build time), and so on. They should be semantically equivalent, but it's not fully guaranteed.
- dd_ 4y ago
  You're not wrong overall, but your example is. In the build environment, the current time is set to the epoch which removes that variability. An upcoming feature of nix is content addressed builds which use the output hashes to determine the store path. This should make it much more transparent which builds are reproducible and which are not.
  - Filligree 4y ago
    > In the build environment, the current time is set to the epoch which removes that variability. The amount of time per build can vary. Embedded timestamps will all be in January 1970, but even a single second difference means the binaries are no longer bit for bit identical -- and that assumes the embedded timestamp isn't microseconds.
    - infogulch 4y ago
      I think you're assuming that the timestamp only starts at the epoch and still advances in real time, which is not necessarily true.
      - Filligree 4y ago
        If it didn't advance, that would cause trouble for some build systems. So yes, I'm assuming it does.
linuxhackerman 4y ago
"reproducible" means you get the same result, bit-for-bit, when running a build in different circumstances. It's trivial to write non-reproducible Nix derivations by placing timestamps, randomly generated data, or similar into the build output, directly or indirectly. Nix is a powerful tool for working towards reproducibility, since it can specify the software side of a build environment more exhaustively than most other package managers out there, and has a language (derivations) that enables repeating builds easily as well as comparing them, but does not guarantee reproducibility on its own. A lot of the reproducibility of nixpkgs is attributable to reproducible build efforts elsewhere, including conventions like SOURCE_DATE_EPOCH. Nix itself does not in fact "freeze the clock" for builds, but many tools that use timestamps as part of a build will respect SOURCE_DATE_EPOCH instead of using the system clock. Many of these valuable developments in this direction are coordinated in the cross-distro https://reproducible-builds.org https://reproducible-builds.org group.
dlahoda 4y ago
hash of build output is not used in hash of derivation and in derivation tree. hence it is not reproducible.

JamesSwift 4y ago

With flakes it is reproducible in a self-contained way.

baby 4y ago

These two words have the same meaning

lalopalota 4y ago
repeatable - do the exact same thing every time reproducable - get the exact same results every time not the same thing
pxc 4y ago
Repeating a non-deterministic process can give you a different result, which makes the results non-reproducible. Nix does provide a higher degree of functional reproducibility (builds will have the same behavior, if not identical bits) than other systems do, though, because of the way it locks down exact dependency versions and build flags, though. In that sense, Nix is 'more reproducible' than competing tools even when both are invoking non-deterministic compilers at the last step.

pushtheenvelope 4y ago

would you mind elaborating on the difference?

0cf8612b2e1e 4y ago

Accurate, but there is only so much to be done on that front when the consumed packages do not strive for reproducibility. As Nix or similar tools bring the lack of bit-for-bit reproducibility to the forefront, I hope build systems start adopting processes that will correct this behavior.

tripdout 4y ago

I mean, its impossible to guarantee full reproducibility under all possible conditions, but Nix does take steps towards promoting it. eg setting all times to UNIX epoch, enforcing purity (so specifying dependencies explicitly instead of fetching them from $PATH, etc).