4 ms·

+100 From what I saw the package definitions are in python. Given python is really an imperative language I don't see how spack could be comparably powerful fo

+100 From what I saw the package definitions are in python. Given python is really an imperative language I don't see how spack could be comparably powerful for definitions and reuse.

stabbles 4y ago
The package definitions are quite declarative even though the language is imperative, see an example here [1]. For hashes, Spack uses the normalized (comments, whitespace etc removed) abstract syntax tree. With Spack it's easier to contribute, you don't have to learn a new language as you need for Nix / Guix. [1] https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/zstd/package.py#L9-L57 https://github.com/spack/spack/blob/develop/var/spack/repos/...
yjftsjthsd-h 4y ago
I could be persuaded to trade a certain amount of power for ease of use
tgamblin 4y ago
The Spack DSL for defining versions, options, etc. is declarative. That part of a package exposes options to the solver and very much allows reuse, in the sense that the same package can be built many different ways, and even with different dependencies. The whole system is parameterized in ways that Nix is not. The imperative part of a Spack package is the build recipe, which AFAICT is not so different from bundling a bash script in a Nix derivation. Just like Nix, it's included in our configuration hash, so if you change the recipe you get a different installation of the software.
- 331c8c71 4y ago
  Thanks for the answer. > That part of a package exposes options ... in the sense that the same package can be built many different ways, and even with different dependencies. It is quite a common pattern in nix to have package build flags and the dependencies as inputs in a derivation. This definitely makes it possible to build in different ways and with different dependencies.
  - tgamblin 4y ago
    Yep! I'm not a nix expert but my understanding is that this is typically done with overlays. So you can, say, swap out openmpi for mpich, but mpich requires a different version of hwloc that conflicts with the one defined in some other branch of your derivation, you're out of luck. In Spack that is handled through dependency resolution. There have been discussions of how to deal with this in Nix, e.g.: https://discourse.nixos.org/t/concept-use-any-package-version/6515 https://discourse.nixos.org/t/concept-use-any-package-versio.... Similarly, for packages like HDF5 that tend to be depended on with options that affect its API (+parallel, ~parallel, etc.), I do not think there are automatic ways to get nix to ensure that your request for hdf5+parallel is consistent with other packages in the graph. Possibly worth a mention: for uarch flags like the ones shown here: https://nixos.wiki/wiki/Build_flags https://nixos.wiki/wiki/Build_flags, the user is responsible for setting them specifically for the requested compiler. We've abstracted that with `archspec` (https://github.com/archspec/archspec https://github.com/archspec/archspec) so that you can just ask for a target and the correct compiler flags are injected. e.g., `target=cascadelake` or `target=zen3`. So if you switch compiler, you do not have to look up these details. See https://tgamblin.github.io/pubs/archspec-canopie-hpc-2020.pdf https://tgamblin.github.io/pubs/archspec-canopie-hpc-2020.pd... for more on how it works. The salient parts of archspec aren't in a python library but in a JSON file (https://github.com/archspec/archspec-json/blob/master/cpu/microarchitectures.json https://github.com/archspec/archspec-json/blob/master/cpu/mi...) so it's something that could probably be incorporated into Nix.
pxc 4y ago
After using it for ~8 years, I rather like Nixlang. At the same time, I don't know Python well or particularly like it, so Spack's DSL just doesn't look as comfy to me. It reminds me of RPM spec macros a little bit, which I guess are fine but I don't love them. I'd rather see Nix learn from Spack than myself jump ship. But I think it would be very silly to dismiss Spack over something like the fact that their DSL is Python-based. It's fine to be partial to what you know, and there are lots of good reasons to prefer Nix today, at well as to think that improving Nix might be better for you than adopting Spack. But I think we should go a bit deeper than this in evaluating competing projects. Nix is about more than just being in some FP club, you know?

+100 From what I saw the package definitions are in python. Given python is really an imperative language I don't see how spack could be comparably powerful for definitions and reuse.

stabbles 4y ago

The package definitions are quite declarative even though the language is imperative, see an example here [1]. For hashes, Spack uses the normalized (comments, whitespace etc removed) abstract syntax tree. With Spack it's easier to contribute, you don't have to learn a new language as you need for Nix / Guix. [1] https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/zstd/package.py#L9-L57 https://github.com/spack/spack/blob/develop/var/spack/repos/...

yjftsjthsd-h 4y ago

I could be persuaded to trade a certain amount of power for ease of use

tgamblin 4y ago

The Spack DSL for defining versions, options, etc. is declarative. That part of a package exposes options to the solver and very much allows reuse, in the sense that the same package can be built many different ways, and even with different dependencies. The whole system is parameterized in ways that Nix is not. The imperative part of a Spack package is the build recipe, which AFAICT is not so different from bundling a bash script in a Nix derivation. Just like Nix, it's included in our configuration hash, so if you change the recipe you get a different installation of the software.

331c8c71 4y ago
Thanks for the answer. > That part of a package exposes options ... in the sense that the same package can be built many different ways, and even with different dependencies. It is quite a common pattern in nix to have package build flags and the dependencies as inputs in a derivation. This definitely makes it possible to build in different ways and with different dependencies.
- tgamblin 4y ago
  Yep! I'm not a nix expert but my understanding is that this is typically done with overlays. So you can, say, swap out openmpi for mpich, but mpich requires a different version of hwloc that conflicts with the one defined in some other branch of your derivation, you're out of luck. In Spack that is handled through dependency resolution. There have been discussions of how to deal with this in Nix, e.g.: https://discourse.nixos.org/t/concept-use-any-package-version/6515 https://discourse.nixos.org/t/concept-use-any-package-versio.... Similarly, for packages like HDF5 that tend to be depended on with options that affect its API (+parallel, ~parallel, etc.), I do not think there are automatic ways to get nix to ensure that your request for hdf5+parallel is consistent with other packages in the graph. Possibly worth a mention: for uarch flags like the ones shown here: https://nixos.wiki/wiki/Build_flags https://nixos.wiki/wiki/Build_flags, the user is responsible for setting them specifically for the requested compiler. We've abstracted that with `archspec` (https://github.com/archspec/archspec https://github.com/archspec/archspec) so that you can just ask for a target and the correct compiler flags are injected. e.g., `target=cascadelake` or `target=zen3`. So if you switch compiler, you do not have to look up these details. See https://tgamblin.github.io/pubs/archspec-canopie-hpc-2020.pdf https://tgamblin.github.io/pubs/archspec-canopie-hpc-2020.pd... for more on how it works. The salient parts of archspec aren't in a python library but in a JSON file (https://github.com/archspec/archspec-json/blob/master/cpu/microarchitectures.json https://github.com/archspec/archspec-json/blob/master/cpu/mi...) so it's something that could probably be incorporated into Nix.

pxc 4y ago

After using it for ~8 years, I rather like Nixlang. At the same time, I don't know Python well or particularly like it, so Spack's DSL just doesn't look as comfy to me. It reminds me of RPM spec macros a little bit, which I guess are fine but I don't love them. I'd rather see Nix learn from Spack than myself jump ship. But I think it would be very silly to dismiss Spack over something like the fact that their DSL is Python-based. It's fine to be partial to what you know, and there are lots of good reasons to prefer Nix today, at well as to think that improving Nix might be better for you than adopting Spack. But I think we should go a bit deeper than this in evaluating competing projects. Nix is about more than just being in some FP club, you know?