4 ms·

That’s how nix guarantees reproducibility. The compiled artifacts can be cached but that didnt quite work out of the box

by maytc 6y ago

throwaway894345 6y ago
Nix guarantees reproducibility, which means anything can be rebuilt from scratch, but that's a very abnormal use case. If it didn't work out of the box, it's a problem with the package scripts (the "derivations"). That said, all of our software tends to bottom out in a bunch of shitty C libraries that are all delicately cobbled together with autotools and cmake, so anything that aspires to reproduce these things is going to have issues. This tends to make Nix difficult to use, because it doesn't have nearly the same investment/manpower (yet) as other package ecosystems that is necessary to wrangle these dependencies into a stable foundation that doesn't leak its underlying havoc to higher levels of the stack.
- microphone987 6y ago
  This comment is really, really confusing. If it builds once Nix, it will build again with Nix... If you can find a derivation that builds on one machine and not on the other there's usually a fundamental difference - either in CPU arch, your nixpkgs config differs (overlays), etc. Or I guess you could write a build script that non-deterministic ally fails, but that has nothing to do with Nix's maturity. Nix ensures the tooling is called the same. I can understand having a non-autotools project that is more difficult to write a derivation for, but "Nix can't wrangle messy libraries" makes absolutely no sense, either from a "make compiling reliable" or anything at use-time. Do you have a specific example in mind that is less hand-wavy?
  - throwaway894345 6y ago
    > If it builds once Nix, it will build again with Nix... That's the aspiration, but it doesn't always pan out. The rate at which you run into problems depends a lot on the packages you use, how much attention is given to them, and how hard it is to reproducibly package them. I've noticed in particular that the Python ecosystem is really fragile. > Do you have a specific example in mind that is less hand-wavy? A specific example that comes to mind was the psycopg2 Python package, which would build on some developers' machines but not on others. This sort of thing happens all the time, usually on macOS, and usually with C packages (sadly, so much of the Python ecosystem is built on C and its shoddy build tooling). I've also found quite a few packages in nixpkgs that simply don't build on macOS, but which presumably build on Linux; however, I forget which ones specifically.

That’s how nix guarantees reproducibility. The compiled artifacts can be cached but that didnt quite work out of the box

by maytc 6y ago

That’s how nix guarantees reproducibility. The compiled artifacts can be cached but that didnt quite work out of the box

throwaway894345 6y ago

Nix guarantees reproducibility, which means anything can be rebuilt from scratch, but that's a very abnormal use case. If it didn't work out of the box, it's a problem with the package scripts (the "derivations"). That said, all of our software tends to bottom out in a bunch of shitty C libraries that are all delicately cobbled together with autotools and cmake, so anything that aspires to reproduce these things is going to have issues. This tends to make Nix difficult to use, because it doesn't have nearly the same investment/manpower (yet) as other package ecosystems that is necessary to wrangle these dependencies into a stable foundation that doesn't leak its underlying havoc to higher levels of the stack.

microphone987 6y ago
This comment is really, really confusing. If it builds once Nix, it will build again with Nix... If you can find a derivation that builds on one machine and not on the other there's usually a fundamental difference - either in CPU arch, your nixpkgs config differs (overlays), etc. Or I guess you could write a build script that non-deterministic ally fails, but that has nothing to do with Nix's maturity. Nix ensures the tooling is called the same. I can understand having a non-autotools project that is more difficult to write a derivation for, but "Nix can't wrangle messy libraries" makes absolutely no sense, either from a "make compiling reliable" or anything at use-time. Do you have a specific example in mind that is less hand-wavy?
- throwaway894345 6y ago
  > If it builds once Nix, it will build again with Nix... That's the aspiration, but it doesn't always pan out. The rate at which you run into problems depends a lot on the packages you use, how much attention is given to them, and how hard it is to reproducibly package them. I've noticed in particular that the Python ecosystem is really fragile. > Do you have a specific example in mind that is less hand-wavy? A specific example that comes to mind was the psycopg2 Python package, which would build on some developers' machines but not on others. This sort of thing happens all the time, usually on macOS, and usually with C packages (sadly, so much of the Python ecosystem is built on C and its shoddy build tooling). I've also found quite a few packages in nixpkgs that simply don't build on macOS, but which presumably build on Linux; however, I forget which ones specifically.