10 ms·

Spack – scientific software package manager for supercomputers, Linux, and macOS

157 points by telotortium 4y ago

ephimetheus 4y ago
I’ve tried to like this more than once, but once you hit some cryptic error somewhere in the build you’re not going to have a great time. Also unfortunately it doesn’t always play nicely with lots of high energy physics software which really leans on using LD_LIBRARY_PATH and running custom executables during builds. I’ve had bad experiences with ROOT and Gaudi with no obvious paths forward for fixing things.
- tgamblin 4y ago
  Sorry to hear this. We have a lot of connections with the high energy physics community, which is why ROOT and Gaudi are even in Spack to begin with. There is an #hep channel on https://slack.spack.io https://slack.spack.io, and we talk to those folks fairly frequently to figure out what we can add to support these codes better. On `LD_LIBRARY_PATH` specifically: we intentionally inject RPATH-ing compiler wrappers into the build to avoid these types of problems, which are on by default for the ROOT and Gaudi builds. Would be curious to hear where the remaining pain points are.
- wdconinc 4y ago
  We have used spack to maintain our 1000-person user environments at the electron-ion collider (including ROOT, Gaudi) for the past two years and couldn't have done it otherwise.
- prpl 4y ago
  Both of those are non-trivial and annoying to say the least, I know ROOT has a few very smart people managing build recipes in conda-forge too.
- gnufx 4y ago
  Yes, I've found it increasingly frustrating after managing a university central HPC system with rpms. (It's a myth that you can't support different versions with rpm or dpkg if necessary.) I was at least convinced of the need for an actual package manager -- i.e. not Easybuild that everyone else favoured -- after the hell of people's environment modules installations and combinatorial explosions. For instance, several times I've had Spack in a state where it had obscure apparently internal errors that I didn't have the energy to try to diagnose, which resulted in throwing everything away and starting again. I found build recipes frequently don't actually work, especially if you're not on x86_64, though architecture accounts for a minority of the failures. You also end up quickly running out of space in ~10GB of home directory, at least if things aren't shared on the system. There is a tension between "reproducible" static configurations and the dynamics you need, e.g. to be able to link instrumentation or acceleration libraries at run time. However, there is a Spack option to use RUNPATH instead of RPATH that I'd have to look up.
prpl 4y ago
I was excited for Spack and used it for some small packages, but it didn’t have a proper binary artifacts then. Ended up settling on conda-forge and it’s okay. If I was doing a large scientific computing project today, I’d probably reach for bazel with some conda rules hooked up to conda-forge to set up third party packages.
klysm 4y ago
I’m naive about HPC, but it’s always surprised me a bit that HPC software seems to be a fairly distinct ecosystem. I’m not doubting it’s for good reasons, but why is a special package manager needed for this domain?
- tgamblin 4y ago
  HPC isn't exactly special, and neither to some extent is Spack. Spack's gotten some attention in talks at, e.g., CppCon. The big differentiator is really the degree to which people want to tune and customize their builds in HPC vs. other communities, and the diversity of hardware that needs to be supported. Things that stand out to me: - different applications' needs to customize the builds of their dependencies - e.g., one app might need HDF5 with MPI support, another might want it built without. Those are two different, incompatible HDF5 builds. - need for specific microarchitecture builds to take advantage of vectorization - need for GPU support (for NVIDIA, AMD, *and* Intel GPUs) - need to use sometimes vendor-specific tuned system libraries like MPI, cray-libsci, mol - need for specific *versions* of dependencies (solvers, mesh libs, etc.) for numerical reproducibility - need to integrate across languages, e.g. C, C++, Fortran, Python, Lua, perl, R, and dare I even say Yorick. Most of these requirements are not so dissimilar from peoples' dev environments in, say, the AI community, where people really want their special version of PyTorch. Where monorepos are common in industry, they really haven't taken off in the distributed, worldwide scientific community, so you get things like Spack that let you keep rebuilding the world -- and all the microcosms in it. So I'd say Spack is not so much a special package manager as a much more general one. You can use it for combinatorial deployments at HPC centers, dev workflows for people dealing with multi-physics and other complex codes, and as sort of a distributed poly repo with lock files. The intent was never to be specific to HPC, and I would love to see broader adoption outside this community.
  - klysm 4y ago
    Thanks for the thoughtful response. That makes a lot of sense and these capabilities are not something you'll find in debian packages. It's not easy to ask for different compilation flags there.
- 4y ago
TheDesolate0 4y ago
[dead]
elcritch 4y ago
Anyone tried using this with ROS2? It has similar issue of using LD_LIBRARY_PATH during building that the HEP packages had. Not to mention its own layer on top of cmake.
arjvik 4y ago
Does this work with the "environment module" system (either the perl `module` or modern luamod `module`) commonly used in HPC, and what does the interop look like if so?
- tgamblin 4y ago
  Yep, it works with Lmod (https://lmod.readthedocs.io https://lmod.readthedocs.io) and environment modules (https://modules.sourceforge.net https://modules.sourceforge.net). See the tutorial on setting up modules for what the integration looks like: * https://spack-tutorial.readthedocs.io/en/latest/tutorial_modules.html https://spack-tutorial.readthedocs.io/en/latest/tutorial_mod... AFAIK environment modules are implemented in TCL, not perl. The original was in C with an embedded TCL interpreter, but when CEA revived the project w/version 4 in 2017, it was all in TCL.
- rekado 4y ago
  This may also be of interest: https://hpc.guix.info/blog/2022/05/back-to-the-future-modules-for-guix-packages/ https://hpc.guix.info/blog/2022/05/back-to-the-future-module... Guix also has a compatibility layer with environment modules, though you likely wouldn't compose environments that mix in Guix stuff with existing modules. It's intended as an output of Guix.
eslaught 4y ago
I'm surprised that no one here has talked about the build times yet. As a new user, the first thing you notice out of the box is that even on a relatively beefy system, you spend ~2 hours getting to a working state, even when you're building a relatively modest set of packages. This is particularly surprising since one of the points of Spack is to integrate with the existing system software, and so much of what you're building already exists and works fine on the system. In the past when I've talked to people about this, the answer I got was "well, set it up once for your team and don't reinstall it all the time". Which is kind of awful from a "cattle, not pets" perspective. Several other design decisions in Spack (like the lack of certain kinds of configuration isolation by default, and the general difficulty of configuring certain kinds of things) make it feel like a "pets" system. I've gotten instructions from people on using Spack to install things that have been impossible to follow because they fiddled with something in their environment and forgot to tell me. The best thing I can say about Spack is that all the other systems are worse. I just wish they could get around to fixing some of this stuff already.
- EntrePrescott 4y ago
  > like the lack of certain kinds of configuration isolation by default could you please explain a little what kind of "configuration isolation" you mean? > and the general difficulty of configuring certain kinds of things like what for example?
  - eslaught 4y ago
    > could you please explain a little what kind of "configuration isolation" you mean? It's been a while so I don't have the whole list off the top of my head, but there are several configuration files that live in $HOME by default unless you set SPACK_DISABLE_LOCAL_CONFIG=1. Those files can e.g., list compilers and software packages, if I recall, among various other settings. > like what for example? I was trying to follow someone's recipe for building something with Spack. I hit some sort of compiler error (sorry, it's been too long, don't remember). And the solution they suggested was to modify compilers.yml by hand to set various things. (Something about setting the right paths for using Cray's compiler wrappers with rocmcc...) That and I had to fuss with packages.yml to configure the system MPI. I remember trying to figure out how use "spack config set" to configure these files automatically, but the syntax was (a) completely undocumented and (b) not actually powerful enough in practice to configure either of those files. So I gave up on that, set SPACK_DISABLE_LOCAL_CONFIG and just saved the modified yaml files into my repository so that I can at least recover my settings if I need them in the future. For advanced use cases, it seems like you basically need to modify these yaml files to get anything done, but the amount of documentation and support for doing this automatically is really lacking. (And the idea that you're supposed to require every user to do this manually is even more insane.)
hkt 4y ago
The name is a bit unfortunate, to say the least: https://en.m.wiktionary.org/wiki/spack https://en.m.wiktionary.org/wiki/spack
spott 4y ago
This sounds great. I'd love to have something like this within nix. Nix is great for full system management. Less so for something like python package management. Its lack of a dependency resolver or multiple versions of a package within the same nixpkgs instance makes it difficult to work with in some (lots of) instances. Generally, I use pipenv or poetry from within a nix flake to define the environment. Spack doesn't seem to cover system management (please correct me if I'm wrong here, but it seems to be out of scope). Combining them: using spack within a flake to define and build a set of dependencies seems like the best of both worlds. Unfortunately, at the moment, it doesn't appear to be available in nixpkgs.
phissenschaft 4y ago
Been using Spack for a while to manage my machine learning package dependencies. It allows me to quickly spin up projects with complex dependencies (my current environment has 329 packages built ...). It's pretty easy to use with containers. It allows me to evaluate and migrate to different PyTorch/CUDA versions easily.
sjsanc 4y ago
Not to be confused with the SWC bundler for JavaScript https://swc.rs/docs/configuration/bundling https://swc.rs/docs/configuration/bundling
Moissanite 4y ago
I encourage anyone to follow the "install your first package" instructions, but pick a real scientific package like WRF or GROMACS. Be prepared to wait a while though...
- anon9874 4y ago
  If you do pick WRF, you can also use a binary cache, as shown in this workshop https://weather.hpcworkshops.com/ https://weather.hpcworkshops.com/ .
  - Moissanite 4y ago
    I haven't tried since the binary cache launched - but over the years I've attempted to use Spack following their "getting started" guide half a dozen times and achieved nothing but long waits and disappointment; part of the reason for recommending it was a sanity check on whether I am just impatient or have unreasonable expectations when it comes to software behaving as documented.
kichimi 4y ago
An unfortunate name, a la Nonce Finance and Git. https://en.m.wiktionary.org/wiki/spack https://en.m.wiktionary.org/wiki/spack
- b800h 4y ago
  Have to say I was fairly surprised by this name. It's a fairly old-fashioned expression, but could well cause some misunderstandings.
  - kichimi 4y ago
    I wouldn't say that, it sees regular use even today.
    - b800h 4y ago
      To be fair, I should ask my kids.
- trollied 4y ago
  This is definitely an international blunder. I’d say it’s more socially acceptable in the UK to call someone a c*t than it is to use this word. Can safely say it’ll get zero UK adoption due to the name. Shame.
  - stonogo 4y ago
    Given that it's in use at Cambridge and several other HPC centers around the UK, I'm going to disagree. I think most people are cosmopolitan enough to set aside the UK's niche insult community to move forward with real work. I doubt there's any single-syllable morpheme which doesn't translate to something offensive somewhere.
    - mellosouls 4y ago
      the UK's niche insult community It's a horrible term that was used for years to denigrate disabled people and that it is used now in cloistered and privileged academic settings is no excuse for an ignorant dismissal like that of the hurt, bigotry and exclusion it conveys.
pharmakom 4y ago
What does this offer over the usual suspects brew, linuxbrew, apt… ?
- throw0101b 4y ago
  Multiple versions of the same package can be handy: e.g., Python 3.6, 3.8. and 3.10 all handled by the package manager and able to switch between them on a per-session/terminal basis (no need to completely disable a particular install: just adjust $PATH and $LIB as needed). Able to create pyenv-like environments which can then be 'exported' and sent to other systems/users having Spack so that development/debugging can be done with the same setup: * https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html https://spack-tutorial.readthedocs.io/en/latest/tutorial_env... Able to use 'internal' packages (spack install openssl) or external packages (link against /usr/lib/libssl.so) on a per item basis: sometimes you want the newest code, but other times you want the OS package so that you can get security updates 'for free'. Most other system do one or the other: Spack allows you to choose.
- stonogo 4y ago
  Spack allows you to have several matrices of the same package installed. So, if you want to install Foobar, and you want one copy built with gcc and linked to openmpi, one copy with llvm and linked to mpich, and three copies with the Intel compiler and three different versions of the Intel mpi libraries,spack can do that for you, and help manage your environment to ensure you are running the stack you intend. It's primary use is in supercomputing to make various optimizations available to users.
- CaliforniaKarl 4y ago
  The examples you listed are designed to install packages on a single system. Spack is designed to install packages in a network-storage path that can be accessed by multiple semi-heterogeneous systems that run the same main architecture.
JorgeGT 4y ago
How does it compares to other HPC software managers such as easybuild?
- throw0101b 4y ago
  From the 7th EasyBuild User Meeting (2022), 'A noob test: Spack "vs" EasyBuild' presentation is available on YouTube: * https://easybuild.io/eum22/#spack-vs-eb https://easybuild.io/eum22/#spack-vs-eb @tgamblin also presented 'The Spack 2022 Roadmap': * https://easybuild.io/eum22/#spack https://easybuild.io/eum22/#spack And will also present at 8th EUM (April): * https://easybuild.io/eum/#spack https://easybuild.io/eum/#spack
  - boegel 4y ago
    Better link for Spack talk at EUM'23: (the 'eum' link is a moving target) There's also an (now a bit outdated) talk compared EasyBuild with Spack and other alternatives like Nix/Guix which I gave a FOSDEM'18: https://archive.fosdem.org/2018/schedule/event/installing_software_for_scientists https://archive.fosdem.org/2018/schedule/event/installing_so... The EasyBuild User Meetings have always been very open to having talks on "competing" tools. Todd (or someone else) giving an update on recent developments in Spack is becoming a tradition (see also https://easybuild.io/eum21/#spack https://easybuild.io/eum21/#spack, https://github.com/easybuilders/easybuild/wiki/5th-EasyBuild-User-Meeting https://github.com/easybuilders/easybuild/wiki/5th-EasyBuild..., etc.)
- tgamblin 4y ago
  If I have to pick one difference, the fundamental one is that Spack does dependency resolution with a solver, while EB does not. So, in Spack, you write a package.py file (like say the one for gromacs: https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/gromacs/package.py https://github.com/spack/spack/blob/develop/var/spack/repos/...), and you can build that package with any of the versions/options/dependencies/etc. expressed in the package. With Easybuild, someone (maybe you?) has to write a specific configuration (easyconfig) for every build you want to do. Want to tweak a version? Write a new easyconfig. The dependencies point at specific versions, too, so now write configs for all the dependencies, and so on. There is a lot of copying/tweaking of file trees in the EB workflow. Aside from that: - Spack has a package database and allows you to query what is installed. - EB does not have "uninstall", just `rm -rf` - Spack builds from source and also handles binary packages (no binaries in EB) - Spack supports environments via `spack.yaml` / `spack.lock`; EB doesn't have that. - Spack doesn't require Lmod to work -- you can just `spack load` or `spack env activate` to use packages. - Spack builds with RPATH by default (much like Nix does) -- dependency libraries are found automatically even if you run an executable straight from its directory. There's more but Spack is a package manager, while EB is really set up to automate a certain type of installation common in HPC.
amelius 4y ago
Does it have an "undo" option?
- stabbles 4y ago
  It doesn't have it, but what sort of undo do you think of? Undo the latest installation on disk? Roll back to the previous environment?
  - tgamblin 4y ago
    I'd love to have both of these, to be honest.
tgamblin 4y ago
Original author here -- since I know that people click the comments to decide whether to read further, here's a (biased?) summary: Spack is aimed at end users, developers, and admins. It's used primarily in HPC at the moment but it would be great to see use outside of the HPC community. Spack supports: 1. Basic installation (spack install foo, foo@3.1.2, foo@3.1.2 +option, etc.) 2. Reproducible environments, via `spack.yaml` and `spack.lock` files that you can version in a repo. You can use this to build a custom software stack, build some dependencies for a project, etc. 3. Building many versions of the same package, e.g. these are perfectly fine and can coexist: - spack install hdf5 - spack install hdf5 %clang - spack install hdf5 %oneapi@2023.2.0 - spack install hdf5@1.18.0 cxxflags="-O3 -fast" target=cascadelake - spack install hdf5 +mpi ^mpich - spack install hdf5 +mpi ^openmpi 4. Both source builds and installation from relocatable binary caches 5. Lots of conditional features in packages that other systems do not offer, e.g.: - injecting compiler flags - building for specific uarch targets - optional dependencies - very fine-grained options (e.g. +/- cuda, set cuda_arch, etc.) - building the same package with different compilers - building the same package with different potentially ABI-incompatible dependencies like MPI or different boost versions. 5. Autogenerating things like container recipes, CI pipelines, Lmod/environment modules, etc. 6. Good support for building against external packages that may already be on your system. At a very high level, Spack has: * Nix's installation model and configuration hashing * Homebrew-like packages, but in a more expressive Python DSL, and with more versions/options * A very powerful dependency resolver that doesn't just pick from a set of available configurations -- it configures your build according to possible configurations. You could think of it like Nix with dependency resolution, but with a nice Python DSL. There is more on the "concretizer" (resolver) and how we've used ASP for it here: * "Using Answer Set Programming for HPC Dependency Solving", https://arxiv.org/abs/2210.08404 https://arxiv.org/abs/2210.08404
- throw0101b 4y ago
  > 6. Good support for building against external packages that may already be on your system. Able to use 'internal' packages (spack install openssl) or external packages (link against /usr/lib/libssl.so) on a per item basis: sometimes you want the newest code, but other times you want the OS package so that you can get security updates 'for free'. Most other system do one or the other: Spack allows you to choose.
evancordell 4y ago
There's a good interview with the author in an episode of The Manifest: https://manifest.fm/11 https://manifest.fm/11 It's been a while since I've listened, but I remember it being pretty interesting.

157 points by telotortium 4y ago

I’ve tried to like this more than once, but once you hit some cryptic error somewhere in the build you’re not going to have a great time. Also unfortunately it doesn’t always play nicely with lots of high energy physics software which really leans on using LD_LIBRARY_PATH and running custom executables during builds. I’ve had bad experiences with ROOT and Gaudi with no obvious paths forward for fixing things.

tgamblin 4y ago
Sorry to hear this. We have a lot of connections with the high energy physics community, which is why ROOT and Gaudi are even in Spack to begin with. There is an #hep channel on https://slack.spack.io https://slack.spack.io, and we talk to those folks fairly frequently to figure out what we can add to support these codes better. On `LD_LIBRARY_PATH` specifically: we intentionally inject RPATH-ing compiler wrappers into the build to avoid these types of problems, which are on by default for the ROOT and Gaudi builds. Would be curious to hear where the remaining pain points are.
wdconinc 4y ago
We have used spack to maintain our 1000-person user environments at the electron-ion collider (including ROOT, Gaudi) for the past two years and couldn't have done it otherwise.
prpl 4y ago
Both of those are non-trivial and annoying to say the least, I know ROOT has a few very smart people managing build recipes in conda-forge too.
gnufx 4y ago
Yes, I've found it increasingly frustrating after managing a university central HPC system with rpms. (It's a myth that you can't support different versions with rpm or dpkg if necessary.) I was at least convinced of the need for an actual package manager -- i.e. not Easybuild that everyone else favoured -- after the hell of people's environment modules installations and combinatorial explosions. For instance, several times I've had Spack in a state where it had obscure apparently internal errors that I didn't have the energy to try to diagnose, which resulted in throwing everything away and starting again. I found build recipes frequently don't actually work, especially if you're not on x86_64, though architecture accounts for a minority of the failures. You also end up quickly running out of space in ~10GB of home directory, at least if things aren't shared on the system. There is a tension between "reproducible" static configurations and the dynamics you need, e.g. to be able to link instrumentation or acceleration libraries at run time. However, there is a Spack option to use RUNPATH instead of RPATH that I'd have to look up.

prpl 4y ago

I was excited for Spack and used it for some small packages, but it didn’t have a proper binary artifacts then. Ended up settling on conda-forge and it’s okay. If I was doing a large scientific computing project today, I’d probably reach for bazel with some conda rules hooked up to conda-forge to set up third party packages.

klysm 4y ago

I’m naive about HPC, but it’s always surprised me a bit that HPC software seems to be a fairly distinct ecosystem. I’m not doubting it’s for good reasons, but why is a special package manager needed for this domain?

tgamblin 4y ago
HPC isn't exactly special, and neither to some extent is Spack. Spack's gotten some attention in talks at, e.g., CppCon. The big differentiator is really the degree to which people want to tune and customize their builds in HPC vs. other communities, and the diversity of hardware that needs to be supported. Things that stand out to me: - different applications' needs to customize the builds of their dependencies - e.g., one app might need HDF5 with MPI support, another might want it built without. Those are two different, incompatible HDF5 builds. - need for specific microarchitecture builds to take advantage of vectorization - need for GPU support (for NVIDIA, AMD, *and* Intel GPUs) - need to use sometimes vendor-specific tuned system libraries like MPI, cray-libsci, mol - need for specific *versions* of dependencies (solvers, mesh libs, etc.) for numerical reproducibility - need to integrate across languages, e.g. C, C++, Fortran, Python, Lua, perl, R, and dare I even say Yorick. Most of these requirements are not so dissimilar from peoples' dev environments in, say, the AI community, where people really want their special version of PyTorch. Where monorepos are common in industry, they really haven't taken off in the distributed, worldwide scientific community, so you get things like Spack that let you keep rebuilding the world -- and all the microcosms in it. So I'd say Spack is not so much a special package manager as a much more general one. You can use it for combinatorial deployments at HPC centers, dev workflows for people dealing with multi-physics and other complex codes, and as sort of a distributed poly repo with lock files. The intent was never to be specific to HPC, and I would love to see broader adoption outside this community.
- klysm 4y ago
  Thanks for the thoughtful response. That makes a lot of sense and these capabilities are not something you'll find in debian packages. It's not easy to ask for different compilation flags there.
4y ago

TheDesolate0 4y ago

[dead]

elcritch 4y ago

Anyone tried using this with ROS2? It has similar issue of using LD_LIBRARY_PATH during building that the HEP packages had. Not to mention its own layer on top of cmake.

arjvik 4y ago

Does this work with the "environment module" system (either the perl `module` or modern luamod `module`) commonly used in HPC, and what does the interop look like if so?

tgamblin 4y ago
Yep, it works with Lmod (https://lmod.readthedocs.io https://lmod.readthedocs.io) and environment modules (https://modules.sourceforge.net https://modules.sourceforge.net). See the tutorial on setting up modules for what the integration looks like: * https://spack-tutorial.readthedocs.io/en/latest/tutorial_modules.html https://spack-tutorial.readthedocs.io/en/latest/tutorial_mod... AFAIK environment modules are implemented in TCL, not perl. The original was in C with an embedded TCL interpreter, but when CEA revived the project w/version 4 in 2017, it was all in TCL.
rekado 4y ago
This may also be of interest: https://hpc.guix.info/blog/2022/05/back-to-the-future-modules-for-guix-packages/ https://hpc.guix.info/blog/2022/05/back-to-the-future-module... Guix also has a compatibility layer with environment modules, though you likely wouldn't compose environments that mix in Guix stuff with existing modules. It's intended as an output of Guix.

eslaught 4y ago

I'm surprised that no one here has talked about the build times yet. As a new user, the first thing you notice out of the box is that even on a relatively beefy system, you spend ~2 hours getting to a working state, even when you're building a relatively modest set of packages. This is particularly surprising since one of the points of Spack is to integrate with the existing system software, and so much of what you're building already exists and works fine on the system. In the past when I've talked to people about this, the answer I got was "well, set it up once for your team and don't reinstall it all the time". Which is kind of awful from a "cattle, not pets" perspective. Several other design decisions in Spack (like the lack of certain kinds of configuration isolation by default, and the general difficulty of configuring certain kinds of things) make it feel like a "pets" system. I've gotten instructions from people on using Spack to install things that have been impossible to follow because they fiddled with something in their environment and forgot to tell me. The best thing I can say about Spack is that all the other systems are worse. I just wish they could get around to fixing some of this stuff already.

EntrePrescott 4y ago
> like the lack of certain kinds of configuration isolation by default could you please explain a little what kind of "configuration isolation" you mean? > and the general difficulty of configuring certain kinds of things like what for example?
- eslaught 4y ago
  > could you please explain a little what kind of "configuration isolation" you mean? It's been a while so I don't have the whole list off the top of my head, but there are several configuration files that live in $HOME by default unless you set SPACK_DISABLE_LOCAL_CONFIG=1. Those files can e.g., list compilers and software packages, if I recall, among various other settings. > like what for example? I was trying to follow someone's recipe for building something with Spack. I hit some sort of compiler error (sorry, it's been too long, don't remember). And the solution they suggested was to modify compilers.yml by hand to set various things. (Something about setting the right paths for using Cray's compiler wrappers with rocmcc...) That and I had to fuss with packages.yml to configure the system MPI. I remember trying to figure out how use "spack config set" to configure these files automatically, but the syntax was (a) completely undocumented and (b) not actually powerful enough in practice to configure either of those files. So I gave up on that, set SPACK_DISABLE_LOCAL_CONFIG and just saved the modified yaml files into my repository so that I can at least recover my settings if I need them in the future. For advanced use cases, it seems like you basically need to modify these yaml files to get anything done, but the amount of documentation and support for doing this automatically is really lacking. (And the idea that you're supposed to require every user to do this manually is even more insane.)

hkt 4y ago

The name is a bit unfortunate, to say the least: https://en.m.wiktionary.org/wiki/spack https://en.m.wiktionary.org/wiki/spack

spott 4y ago

This sounds great. I'd love to have something like this within nix. Nix is great for full system management. Less so for something like python package management. Its lack of a dependency resolver or multiple versions of a package within the same nixpkgs instance makes it difficult to work with in some (lots of) instances. Generally, I use pipenv or poetry from within a nix flake to define the environment. Spack doesn't seem to cover system management (please correct me if I'm wrong here, but it seems to be out of scope). Combining them: using spack within a flake to define and build a set of dependencies seems like the best of both worlds. Unfortunately, at the moment, it doesn't appear to be available in nixpkgs.

phissenschaft 4y ago

Been using Spack for a while to manage my machine learning package dependencies. It allows me to quickly spin up projects with complex dependencies (my current environment has 329 packages built ...). It's pretty easy to use with containers. It allows me to evaluate and migrate to different PyTorch/CUDA versions easily.

sjsanc 4y ago

Not to be confused with the SWC bundler for JavaScript https://swc.rs/docs/configuration/bundling https://swc.rs/docs/configuration/bundling

Moissanite 4y ago

I encourage anyone to follow the "install your first package" instructions, but pick a real scientific package like WRF or GROMACS. Be prepared to wait a while though...

anon9874 4y ago
If you do pick WRF, you can also use a binary cache, as shown in this workshop https://weather.hpcworkshops.com/ https://weather.hpcworkshops.com/ .
- Moissanite 4y ago
  I haven't tried since the binary cache launched - but over the years I've attempted to use Spack following their "getting started" guide half a dozen times and achieved nothing but long waits and disappointment; part of the reason for recommending it was a sanity check on whether I am just impatient or have unreasonable expectations when it comes to software behaving as documented.

kichimi 4y ago

An unfortunate name, a la Nonce Finance and Git. https://en.m.wiktionary.org/wiki/spack https://en.m.wiktionary.org/wiki/spack

b800h 4y ago
Have to say I was fairly surprised by this name. It's a fairly old-fashioned expression, but could well cause some misunderstandings.
- kichimi 4y ago
  I wouldn't say that, it sees regular use even today.
  - b800h 4y ago
    To be fair, I should ask my kids.
trollied 4y ago
This is definitely an international blunder. I’d say it’s more socially acceptable in the UK to call someone a c*t than it is to use this word. Can safely say it’ll get zero UK adoption due to the name. Shame.
- stonogo 4y ago
  Given that it's in use at Cambridge and several other HPC centers around the UK, I'm going to disagree. I think most people are cosmopolitan enough to set aside the UK's niche insult community to move forward with real work. I doubt there's any single-syllable morpheme which doesn't translate to something offensive somewhere.
  - mellosouls 4y ago
    the UK's niche insult community It's a horrible term that was used for years to denigrate disabled people and that it is used now in cloistered and privileged academic settings is no excuse for an ignorant dismissal like that of the hurt, bigotry and exclusion it conveys.

pharmakom 4y ago

What does this offer over the usual suspects brew, linuxbrew, apt… ?

throw0101b 4y ago
Multiple versions of the same package can be handy: e.g., Python 3.6, 3.8. and 3.10 all handled by the package manager and able to switch between them on a per-session/terminal basis (no need to completely disable a particular install: just adjust $PATH and $LIB as needed). Able to create pyenv-like environments which can then be 'exported' and sent to other systems/users having Spack so that development/debugging can be done with the same setup: * https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html https://spack-tutorial.readthedocs.io/en/latest/tutorial_env... Able to use 'internal' packages (spack install openssl) or external packages (link against /usr/lib/libssl.so) on a per item basis: sometimes you want the newest code, but other times you want the OS package so that you can get security updates 'for free'. Most other system do one or the other: Spack allows you to choose.
stonogo 4y ago
Spack allows you to have several matrices of the same package installed. So, if you want to install Foobar, and you want one copy built with gcc and linked to openmpi, one copy with llvm and linked to mpich, and three copies with the Intel compiler and three different versions of the Intel mpi libraries,spack can do that for you, and help manage your environment to ensure you are running the stack you intend. It's primary use is in supercomputing to make various optimizations available to users.
CaliforniaKarl 4y ago
The examples you listed are designed to install packages on a single system. Spack is designed to install packages in a network-storage path that can be accessed by multiple semi-heterogeneous systems that run the same main architecture.

JorgeGT 4y ago

How does it compares to other HPC software managers such as easybuild?

throw0101b 4y ago
From the 7th EasyBuild User Meeting (2022), 'A noob test: Spack "vs" EasyBuild' presentation is available on YouTube: * https://easybuild.io/eum22/#spack-vs-eb https://easybuild.io/eum22/#spack-vs-eb @tgamblin also presented 'The Spack 2022 Roadmap': * https://easybuild.io/eum22/#spack https://easybuild.io/eum22/#spack And will also present at 8th EUM (April): * https://easybuild.io/eum/#spack https://easybuild.io/eum/#spack
- boegel 4y ago
  Better link for Spack talk at EUM'23: (the 'eum' link is a moving target) There's also an (now a bit outdated) talk compared EasyBuild with Spack and other alternatives like Nix/Guix which I gave a FOSDEM'18: https://archive.fosdem.org/2018/schedule/event/installing_software_for_scientists https://archive.fosdem.org/2018/schedule/event/installing_so... The EasyBuild User Meetings have always been very open to having talks on "competing" tools. Todd (or someone else) giving an update on recent developments in Spack is becoming a tradition (see also https://easybuild.io/eum21/#spack https://easybuild.io/eum21/#spack, https://github.com/easybuilders/easybuild/wiki/5th-EasyBuild-User-Meeting https://github.com/easybuilders/easybuild/wiki/5th-EasyBuild..., etc.)
tgamblin 4y ago
If I have to pick one difference, the fundamental one is that Spack does dependency resolution with a solver, while EB does not. So, in Spack, you write a package.py file (like say the one for gromacs: https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/gromacs/package.py https://github.com/spack/spack/blob/develop/var/spack/repos/...), and you can build that package with any of the versions/options/dependencies/etc. expressed in the package. With Easybuild, someone (maybe you?) has to write a specific configuration (easyconfig) for every build you want to do. Want to tweak a version? Write a new easyconfig. The dependencies point at specific versions, too, so now write configs for all the dependencies, and so on. There is a lot of copying/tweaking of file trees in the EB workflow. Aside from that: - Spack has a package database and allows you to query what is installed. - EB does not have "uninstall", just `rm -rf` - Spack builds from source and also handles binary packages (no binaries in EB) - Spack supports environments via `spack.yaml` / `spack.lock`; EB doesn't have that. - Spack doesn't require Lmod to work -- you can just `spack load` or `spack env activate` to use packages. - Spack builds with RPATH by default (much like Nix does) -- dependency libraries are found automatically even if you run an executable straight from its directory. There's more but Spack is a package manager, while EB is really set up to automate a certain type of installation common in HPC.

amelius 4y ago

Does it have an "undo" option?

stabbles 4y ago
It doesn't have it, but what sort of undo do you think of? Undo the latest installation on disk? Roll back to the previous environment?
- tgamblin 4y ago
  I'd love to have both of these, to be honest.

tgamblin 4y ago

Original author here -- since I know that people click the comments to decide whether to read further, here's a (biased?) summary: Spack is aimed at end users, developers, and admins. It's used primarily in HPC at the moment but it would be great to see use outside of the HPC community. Spack supports: 1. Basic installation (spack install foo, foo@3.1.2, foo@3.1.2 +option, etc.) 2. Reproducible environments, via `spack.yaml` and `spack.lock` files that you can version in a repo. You can use this to build a custom software stack, build some dependencies for a project, etc. 3. Building many versions of the same package, e.g. these are perfectly fine and can coexist: - spack install hdf5 - spack install hdf5 %clang - spack install hdf5 %oneapi@2023.2.0 - spack install hdf5@1.18.0 cxxflags="-O3 -fast" target=cascadelake - spack install hdf5 +mpi ^mpich - spack install hdf5 +mpi ^openmpi 4. Both source builds and installation from relocatable binary caches 5. Lots of conditional features in packages that other systems do not offer, e.g.: - injecting compiler flags - building for specific uarch targets - optional dependencies - very fine-grained options (e.g. +/- cuda, set cuda_arch, etc.) - building the same package with different compilers - building the same package with different potentially ABI-incompatible dependencies like MPI or different boost versions. 5. Autogenerating things like container recipes, CI pipelines, Lmod/environment modules, etc. 6. Good support for building against external packages that may already be on your system. At a very high level, Spack has: * Nix's installation model and configuration hashing * Homebrew-like packages, but in a more expressive Python DSL, and with more versions/options * A very powerful dependency resolver that doesn't just pick from a set of available configurations -- it configures your build according to possible configurations. You could think of it like Nix with dependency resolution, but with a nice Python DSL. There is more on the "concretizer" (resolver) and how we've used ASP for it here: * "Using Answer Set Programming for HPC Dependency Solving", https://arxiv.org/abs/2210.08404 https://arxiv.org/abs/2210.08404

throw0101b 4y ago
> 6. Good support for building against external packages that may already be on your system. Able to use 'internal' packages (spack install openssl) or external packages (link against /usr/lib/libssl.so) on a per item basis: sometimes you want the newest code, but other times you want the OS package so that you can get security updates 'for free'. Most other system do one or the other: Spack allows you to choose.

evancordell 4y ago

There's a good interview with the author in an episode of The Manifest: https://manifest.fm/11 https://manifest.fm/11 It's been a while since I've listened, but I remember it being pretty interesting.