6 ms·

I agree with everything but the last statement. This all comes down to: do you consider memoization to be state. I predict people's answers to this question wi

I agree with everything but the last statement. This all comes down to: do you consider memoization to be state. I predict people's answers to this question will come from experience with memoization. Here's mine: I kept trying to get nix to build tensorflow locally, so that I would get the avx512 benefits of the big, but gpu-less machine I had. I hadn't realized some other derivation had already downloaded tensorflow from online cache, so didn't have avx512 enabled. I kept making shells, trying tensorflow, seeing it doesn't have support. The solution was to tell nix to disregard the nix store, in order to force the local build. This experience has left me with the concrete feeling that the nix store is full-on state, and I the user must be aware of it.

hamandcheese 3y ago
Yes, you have unfortunately discovered that sometimes the hardware itself is an input that isn't always captured explicitly, but also isn't controlled for with sandboxing. Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically. I hope that issues like this get better over time thanks to projects like Trustix, which would make non-reproducibility like this more apparent.
- lloeki 3y ago
  > Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically Looks like it sets e.g avx2 on the flags, forcing the package to be most compatible, thus removing the hardware state (the builder may or may not have avx512, ideally Nix packages should remove hardware autodetection to make it pure and consistent in face of cross-compiling). It should indeed have an input in some way, e.g to add more flags. Then AIUI (still learning Nix) one would be able to call the package function with that input from the dependent package function, thus defining another package than the default one, which would be reified as its own specific derivation for that package to depend on. https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/tensorflow/default.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/developmen...
c0balt 3y ago
I think part of the problem ist that derivation hashs sometimes don't fully cover the intermediate states of a derivation during the build process. This might lead to two hashs pointing to effectively two different configurations. I've had that experience in particular with non-reproducible derivations. Even one of those in the store will make the store, or at least a subset of it, a state.
qazxcvbnm 3y ago
Hopefully this sort of issue will be addressed more satisfactorily when content-addressed stores find general usage.
dezgeg 3y ago
> The solution was to tell nix to disregard the nix store, in order to force the local build. If this actually led to avx512 being enabled in the package, then that's a bug. Nix builds should not be dependant on the machine doing the compilation, all such autodetection should be disabled via configure flag or patched out. Then, the right way to enable avx512 would be to pass some 'enable avx512 please' flag to the package's configure flags. Which would then trigger recompilation, without any 'disregard the nix store, in order to force the local build' options.
- omnicognate 3y ago
  It is, but the possibility of such bugs is a downside to the approach (not saying it's a showstopper but it is a negative). Some of the nastiest software problems to track down are the ones that cause some fundamental assumption everything rests on (often a cache keying assumption!) to be broken, making everything behave wrong, including the tools you're supposed to use to track down problems. If you're going to build an entire system on an assumption of referential transparency you want to be able to guarantee that everything really is referentially transparent, and one valid criticism of nix is that it can't really enforce that in all cases.
  - tadfisher 3y ago
    There is work being done to address outputs by their content hash instead of xor-ing their input hashes, which in theory should eliminate this problem.
chisquared 3y ago
Huh, is this what Nix users mean by “impure”? If so, then it makes so much more sense to me now.
- rgoulter 3y ago
  Yes, but 'pure' might mean different things at different levels. In a programming language, 'pure' would just refer to "function where you get the same output from the same input; no side effects". In parent's case, there was an 'impurity' such that the package is meaningfully different depending on the machine it was built on.

hamandcheese 3y ago

Yes, you have unfortunately discovered that sometimes the hardware itself is an input that isn't always captured explicitly, but also isn't controlled for with sandboxing. Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically. I hope that issues like this get better over time thanks to projects like Trustix, which would make non-reproducibility like this more apparent.

lloeki 3y ago
> Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically Looks like it sets e.g avx2 on the flags, forcing the package to be most compatible, thus removing the hardware state (the builder may or may not have avx512, ideally Nix packages should remove hardware autodetection to make it pure and consistent in face of cross-compiling). It should indeed have an input in some way, e.g to add more flags. Then AIUI (still learning Nix) one would be able to call the package function with that input from the dependent package function, thus defining another package than the default one, which would be reified as its own specific derivation for that package to depend on. https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/tensorflow/default.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/developmen...

c0balt 3y ago

I think part of the problem ist that derivation hashs sometimes don't fully cover the intermediate states of a derivation during the build process. This might lead to two hashs pointing to effectively two different configurations. I've had that experience in particular with non-reproducible derivations. Even one of those in the store will make the store, or at least a subset of it, a state.

qazxcvbnm 3y ago

Hopefully this sort of issue will be addressed more satisfactorily when content-addressed stores find general usage.

dezgeg 3y ago

> The solution was to tell nix to disregard the nix store, in order to force the local build. If this actually led to avx512 being enabled in the package, then that's a bug. Nix builds should not be dependant on the machine doing the compilation, all such autodetection should be disabled via configure flag or patched out. Then, the right way to enable avx512 would be to pass some 'enable avx512 please' flag to the package's configure flags. Which would then trigger recompilation, without any 'disregard the nix store, in order to force the local build' options.

omnicognate 3y ago
It is, but the possibility of such bugs is a downside to the approach (not saying it's a showstopper but it is a negative). Some of the nastiest software problems to track down are the ones that cause some fundamental assumption everything rests on (often a cache keying assumption!) to be broken, making everything behave wrong, including the tools you're supposed to use to track down problems. If you're going to build an entire system on an assumption of referential transparency you want to be able to guarantee that everything really is referentially transparent, and one valid criticism of nix is that it can't really enforce that in all cases.
- tadfisher 3y ago
  There is work being done to address outputs by their content hash instead of xor-ing their input hashes, which in theory should eliminate this problem.

chisquared 3y ago

Huh, is this what Nix users mean by “impure”? If so, then it makes so much more sense to me now.

rgoulter 3y ago
Yes, but 'pure' might mean different things at different levels. In a programming language, 'pure' would just refer to "function where you get the same output from the same input; no side effects". In parent's case, there was an 'impurity' such that the package is meaningfully different depending on the machine it was built on.