5 ms·

> Take a look at nix-shell scripts. Much simpler, lightweight and faster alternative to Docker and friends for being able to build a project forever. As resear

by cge 3y ago

> Take a look at nix-shell scripts. Much simpler, lightweight and faster alternative to Docker and friends for being able to build a project forever. As researchers using nix derivations to preserve usability of code meant to be archived with our papers, we found that it was not particularly effective for this task. Even with nixpkgs pinning, our code failed to compile after only a few years. Outside of nixos itself, it does not appear that the build environment is the same; aspects of compilation change, and we ended up needing to change compiler flags a few years later, while expecting that we might need to again at some point. Overall nix has been quite disappointing for us from an archival standpoint.

Aerbil313 3y ago
Nix-shell scripts are ad-hoc, what nix does here is to provide an environment. Nix derivations are different and can be fully declarative and pure. Nixpkgs pinning does not mean specific nixpkgs commit pinning or pinning a specific version of a package. You can do these as well. I wonder what was the underlying reason for your condition.
jolux 3y ago
Just curious, have you written anything up about these experiences with Nix? I've been learning it recently and the reproducibility angle is a huge part of why, if it's oversold then I feel like I should reevaluate how much time I'm putting into it.
deredede 3y ago
Things shouldn't change if the nixpkgs version didn't change, you didn't disable the sandbox, and you don't have master branches of random repos as dependencies. If that's the case, sounds like a major nix bug. If the sandbox is disabled (as it is by default outside of nixos I believe) then yes, you need to be very careful that there is no outside state that leaks into your build (usually through environment variables, PATH but not only - GCC is a big offender here), which sounds like what happened in your case? For archival purposes you really should be enabling the sandbox.
- cge 3y ago
  Thanks: it does seem like the lack of sandboxing is the major problem here. A difficulty in this sort of archiving is that we're not trying to make it so that we can build the code: we're trying to make it so that any reader in the future can. Close to zero of those readers are likely to be using nix for anything else, or to have any experience with nix (or any other approach we might take). Our approach was to make a script for nix-shell that would put the readers in an environment where they could run the code, compiling specific (old) versions of two research software packages. But I think the problem here is that in writing the derivations here, we didn't put gcc in the build inputs. So the system's compiler ended up being used, and with the old, finicky code, compiler flags needed to change. A frustration here is that it appears sandboxing can't be enabled for nix-shell, a problem that has been known for close to a decade [2]. But in our case, if we can get the build inputs to be reasonably complete on our own, it should improve the situation for our readers in the future. [1]: https://github.com/DNA-and-Natural-Algorithms-Group/SST-sequence-designer https://github.com/DNA-and-Natural-Algorithms-Group/SST-sequ... [2]: https://github.com/NixOS/nix/issues/903 https://github.com/NixOS/nix/issues/903
gray_-_wolf 3y ago
You might give GNU Guix a try, combination of guix time-machine and guix shell -C should give you the exact same environment each time without fear of being affected by the rest of the system (with the obvious exception of the linux kernel). And it seems to be popular in science circles.
ParetoOptimal 3y ago
> Even with nixpkgs pinning, our code failed to compile after only a few years. Outside of nixos itself, it does not appear that the build environment is the same; aspects of compilation change, and we ended up needing to change compiler flags a few years later, while expecting that we might need to again at some point. Was this using flakes or no? Since you mention pinning, I'm guessing not. An experience report, even if brief, would be very valuable to a Nix/NixOS community working seriously on reproducibility.

by cge 3y ago

Aerbil313 3y ago

Nix-shell scripts are ad-hoc, what nix does here is to provide an environment. Nix derivations are different and can be fully declarative and pure. Nixpkgs pinning does not mean specific nixpkgs commit pinning or pinning a specific version of a package. You can do these as well. I wonder what was the underlying reason for your condition.

jolux 3y ago

Just curious, have you written anything up about these experiences with Nix? I've been learning it recently and the reproducibility angle is a huge part of why, if it's oversold then I feel like I should reevaluate how much time I'm putting into it.

deredede 3y ago

Things shouldn't change if the nixpkgs version didn't change, you didn't disable the sandbox, and you don't have master branches of random repos as dependencies. If that's the case, sounds like a major nix bug. If the sandbox is disabled (as it is by default outside of nixos I believe) then yes, you need to be very careful that there is no outside state that leaks into your build (usually through environment variables, PATH but not only - GCC is a big offender here), which sounds like what happened in your case? For archival purposes you really should be enabling the sandbox.

cge 3y ago
Thanks: it does seem like the lack of sandboxing is the major problem here. A difficulty in this sort of archiving is that we're not trying to make it so that we can build the code: we're trying to make it so that any reader in the future can. Close to zero of those readers are likely to be using nix for anything else, or to have any experience with nix (or any other approach we might take). Our approach was to make a script for nix-shell that would put the readers in an environment where they could run the code, compiling specific (old) versions of two research software packages. But I think the problem here is that in writing the derivations here, we didn't put gcc in the build inputs. So the system's compiler ended up being used, and with the old, finicky code, compiler flags needed to change. A frustration here is that it appears sandboxing can't be enabled for nix-shell, a problem that has been known for close to a decade [2]. But in our case, if we can get the build inputs to be reasonably complete on our own, it should improve the situation for our readers in the future. [1]: https://github.com/DNA-and-Natural-Algorithms-Group/SST-sequence-designer https://github.com/DNA-and-Natural-Algorithms-Group/SST-sequ... [2]: https://github.com/NixOS/nix/issues/903 https://github.com/NixOS/nix/issues/903

gray_-_wolf 3y ago

You might give GNU Guix a try, combination of guix time-machine and guix shell -C should give you the exact same environment each time without fear of being affected by the rest of the system (with the obvious exception of the linux kernel). And it seems to be popular in science circles.

ParetoOptimal 3y ago

> Even with nixpkgs pinning, our code failed to compile after only a few years. Outside of nixos itself, it does not appear that the build environment is the same; aspects of compilation change, and we ended up needing to change compiler flags a few years later, while expecting that we might need to again at some point. Was this using flakes or no? Since you mention pinning, I'm guessing not. An experience report, even if brief, would be very valuable to a Nix/NixOS community working seriously on reproducibility.