3 ms·

Language specific package managers seem to actually work in real life and are cross platform, e.g. npm. But I'm not aware of anything for e.g. C++ which would o

by nawitus 11y ago

Language specific package managers seem to actually work in real life and are cross platform, e.g. npm. But I'm not aware of anything for e.g. C++ which would offer the features of npm (mainly installing dependencies with a single command and supporting GNU/Linux, OS X and Windows). Is there a single non-language specific package manager which is cross platform, and supports a large percentage of packages for a given programming language? Note that I'm not saying npm is excellent, but I'm not aware of a single successful language-independent package manager. If there are none, why?

gcr 11y ago
Conda aspires to be exactly that. It attempts to package the entire Python scientific stack, but its general purpose nature lets you package other kinds of software as well (for instance, conda can replace compiling arbitrary C programs from source and installing them into your home folder)
Arcsech 11y ago
The aforementioned GNU Guix and the similar Nix project are exactly that: Cross-platform (mostly, at least Nix requires a unixy layer on Windows) language-independent package managers. One could argue that they aren't "successful" (in that they don't have all the packages you might want), but they are still quite new and are pretty different from traditional package managers. I believe that Nix/Guix are the future, but a future that will take some time to become widespread.
- Touche 11y ago
  > Cross-platform (mostly, at least Nix requires a unixy layer on Windows) Not good enough, language package managers must work on Windows, with a regular command prompt.
  - davexunit 11y ago
    Windows is hopeless. If Windows is a must, you really cannot do much better than a package manager per language that you use. No binary reproducibility, no transactional management, no provenance, no thanks.
davexunit 11y ago
>'m not aware of a single successful language-independent package manager Ummmm... dpkg and yum? You know, the package managers that manage the software on everyone's production servers. Or do these not count because they don't work on Windows or something?
- spinningarrow 11y ago
  Yes, his previous sentence specifically mentioned 'cross-platform'.
tgamblin 11y ago
Spack [1,2] is our attempt to do this for HPC, where we have two things going on: 1) Our applications aren't single-language. They include Fortran, Python, C, C++, Yorick (!), Lua, etc. etc. 2) We have a major combinatorial versioning problem, because HPC clusters (and scientific software in general) rely on ABI-incompatible libraries like MPI, and MANY different compilers. Spack is build-from-source and takes a lot of inspiration from Nix and its descendants, but it adds support for things like swapping out compilers/MPI versions/etc., as well as a syntax for composing new builds of things on the fly That is, you can build something with a slightly different version of a dependency without ever editing a package file. Our users to be able to build a package with 6 different compilers and 3 different MPI library versions, and have their 18 versions coexist on the same system. This has been super helpful for testing production code at LLNL, because we have to run our stuff on other peoples' often very different clusters. For instance, at LANL, we prefer PGI compilers. At LLNL we tend to like Intel or Clang. One reason languages tend to grow their own package managers is that there are all kinds of peculiarities about how different languages manage modules/extensions. Have you seen how many different ways there are to deploy Python modules? It's very hard to have multiple versions of things coexisting in the same Python env, and things like pkg_resources require editing client code [3]. Ew. Spack tries to address this by allowing different language packages to implement their own module activation/deactivation logic, so you can kind of have a language-specfic package manager for each install of the language. I'd rather have something more like Nix's profiles, though. I wouldn't say that the project is huge yet, but we've had some interest from mostly other HPC sites. It would be cool if the project caught on in the "real" world, too. [1] https://github.com/llnl/spack https://github.com/llnl/spack [2] http://www.computer.org/csdl/proceedings/sc/2015/3723/00/2807623.pdf http://www.computer.org/csdl/proceedings/sc/2015/3723/00/280... [3] http://stackoverflow.com/questions/5265731 http://stackoverflow.com/questions/5265731
- civodul 11y ago
  I find that Spack in fact takes very little inspiration from functional package managers, Nix and GNU Guix. Dependencies are specified in a loose fashion which, on one hand, gives a lot of flexibility, but on the other hand is detrimental to reproducibility. More importantly, part of the dependency graph, including compilers, is treated specially and essentialy considered outside of the scope of Spack. This seriously hinders reproducibility, as we tried to explain in https://hal.inria.fr/hal-01161771/en https://hal.inria.fr/hal-01161771/en .

by nawitus 11y ago

gcr 11y ago

Conda aspires to be exactly that. It attempts to package the entire Python scientific stack, but its general purpose nature lets you package other kinds of software as well (for instance, conda can replace compiling arbitrary C programs from source and installing them into your home folder)

Arcsech 11y ago

The aforementioned GNU Guix and the similar Nix project are exactly that: Cross-platform (mostly, at least Nix requires a unixy layer on Windows) language-independent package managers. One could argue that they aren't "successful" (in that they don't have all the packages you might want), but they are still quite new and are pretty different from traditional package managers. I believe that Nix/Guix are the future, but a future that will take some time to become widespread.

Touche 11y ago
> Cross-platform (mostly, at least Nix requires a unixy layer on Windows) Not good enough, language package managers must work on Windows, with a regular command prompt.
- davexunit 11y ago
  Windows is hopeless. If Windows is a must, you really cannot do much better than a package manager per language that you use. No binary reproducibility, no transactional management, no provenance, no thanks.

davexunit 11y ago

>'m not aware of a single successful language-independent package manager Ummmm... dpkg and yum? You know, the package managers that manage the software on everyone's production servers. Or do these not count because they don't work on Windows or something?

spinningarrow 11y ago
Yes, his previous sentence specifically mentioned 'cross-platform'.

tgamblin 11y ago

Spack [1,2] is our attempt to do this for HPC, where we have two things going on: 1) Our applications aren't single-language. They include Fortran, Python, C, C++, Yorick (!), Lua, etc. etc. 2) We have a major combinatorial versioning problem, because HPC clusters (and scientific software in general) rely on ABI-incompatible libraries like MPI, and MANY different compilers. Spack is build-from-source and takes a lot of inspiration from Nix and its descendants, but it adds support for things like swapping out compilers/MPI versions/etc., as well as a syntax for composing new builds of things on the fly That is, you can build something with a slightly different version of a dependency without ever editing a package file. Our users to be able to build a package with 6 different compilers and 3 different MPI library versions, and have their 18 versions coexist on the same system. This has been super helpful for testing production code at LLNL, because we have to run our stuff on other peoples' often very different clusters. For instance, at LANL, we prefer PGI compilers. At LLNL we tend to like Intel or Clang. One reason languages tend to grow their own package managers is that there are all kinds of peculiarities about how different languages manage modules/extensions. Have you seen how many different ways there are to deploy Python modules? It's very hard to have multiple versions of things coexisting in the same Python env, and things like pkg_resources require editing client code [3]. Ew. Spack tries to address this by allowing different language packages to implement their own module activation/deactivation logic, so you can kind of have a language-specfic package manager for each install of the language. I'd rather have something more like Nix's profiles, though. I wouldn't say that the project is huge yet, but we've had some interest from mostly other HPC sites. It would be cool if the project caught on in the "real" world, too. [1] https://github.com/llnl/spack https://github.com/llnl/spack [2] http://www.computer.org/csdl/proceedings/sc/2015/3723/00/2807623.pdf http://www.computer.org/csdl/proceedings/sc/2015/3723/00/280... [3] http://stackoverflow.com/questions/5265731 http://stackoverflow.com/questions/5265731

civodul 11y ago
I find that Spack in fact takes very little inspiration from functional package managers, Nix and GNU Guix. Dependencies are specified in a loose fashion which, on one hand, gives a lot of flexibility, but on the other hand is detrimental to reproducibility. More importantly, part of the dependency graph, including compilers, is treated specially and essentialy considered outside of the scope of Spack. This seriously hinders reproducibility, as we tried to explain in https://hal.inria.fr/hal-01161771/en https://hal.inria.fr/hal-01161771/en .