6 ms·

>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform deta

>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implement similar systems in their projects, or realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers.

munificent 11y ago
> Language-specific package manager developers would do well to either implement similar systems in their projects This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. It's a challenge because constraint solving is NP-complete, but it works very well in practice. [1]: https://pub.dartlang.org/ https://pub.dartlang.org/ > realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers. The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.
- davexunit 11y ago
  >This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. Constraint solving is the status quo, and I am strongly opposed to it. Rather than saying "I want foo >= 1.0" in a package manager, you should have the build system test the environment for the features you need. This is what the Autotools and other build systems that have been around for decades do. This way, the project isn't tied to a single package manager and, in the case of free software, greatly eases the burden on package maintainers that want to add your software to their distribution. Functional package management doesn't need constraint solving because every package in the system precisely describes itself: exactly which dependencies are needed for build-time and runtime (which precisely describe themselves, recursively, all the way down to libc), precisely which source code (tarball/directory/whatever + checksum), and the exact build script that turns that source code into a (hopefully bit-reproducible) binary. There's no constraint solving to determine what the dependency tree is, it's already been encoded in the package objects themselves. This is what any robust package manager should enable, but the only two that I know of that do this are Nix and GNU Guix. >The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes. A programming language should not depend on any specific package manager. This is what build systems are for! More and more language communities conflate the two, but this is a huge mistake that we are paying for by making our software nearly impossible to reproduce from source code. [0] Everyone just uses pre-built binaries full of circular dependencies and reproducibility issues because things are so tangled that no one actually knows how to build anything from source anymore. I think the value of language-specific package managers lies in providing an easy way to fetch pure (no native code) modules to faciliate simple code sharing and helping newcomers get bootstrapped quickly. However, for serious software development and deployment they are terrible and we desperately need better tools. I think functional package managers are the tools we need, as they have greatly simplified software building and deployment for me whilst also greatly increasing the reliability of the systems that use them thanks to transactional upgrades and rollbacks. [0] http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html http://www.vitavonni.de/blog/201503/2015031201-the-sad-state...
  - nawitus 11y ago
    Language specific package managers seem to actually work in real life and are cross platform, e.g. npm. But I'm not aware of anything for e.g. C++ which would offer the features of npm (mainly installing dependencies with a single command and supporting GNU/Linux, OS X and Windows). Is there a single non-language specific package manager which is cross platform, and supports a large percentage of packages for a given programming language? Note that I'm not saying npm is excellent, but I'm not aware of a single successful language-independent package manager. If there are none, why?
    - gcr 11y ago
      Conda aspires to be exactly that. It attempts to package the entire Python scientific stack, but its general purpose nature lets you package other kinds of software as well (for instance, conda can replace compiling arbitrary C programs from source and installing them into your home folder)
    - Arcsech 11y ago
      The aforementioned GNU Guix and the similar Nix project are exactly that: Cross-platform (mostly, at least Nix requires a unixy layer on Windows) language-independent package managers. One could argue that they aren't "successful" (in that they don't have all the packages you might want), but they are still quite new and are pretty different from traditional package managers. I believe that Nix/Guix are the future, but a future that will take some time to become widespread.
      - Touche 11y ago
        > Cross-platform (mostly, at least Nix requires a unixy layer on Windows) Not good enough, language package managers must work on Windows, with a regular command prompt.
        davexunit 11y ago
        Windows is hopeless. If Windows is a must, you really cannot do much better than a package manager per language that you use. No binary reproducibility, no transactional management, no provenance, no thanks.
alextgordon 11y ago
When we ship node modules to be embedded into Chocolat.app, we have to strip out all the tests and documentation because otherwise it bloats the download size. And the computer it eventually runs on may not be connected to the internet. The nice thing about node is that it's quite small (3.4MB zipped). We don't have to ship npm with the app, and certainly an OS level package manager would be way too large!
- nawitus 11y ago
  Each package should strip tests and documentation from the package. Of course, many don't..
  - tracker1 11y ago
    True enough... it's not always easy to consider as creating an npm module and building/pushing to NPM is really easy to do.
paulyg 11y ago
This is also what Composer (package manager for PHP) did. They ported openSUSE's libzypp to PHP to solve the dependency management problem. After resolving versions the dependencies are installed in a flat list.

munificent 11y ago

> Language-specific package manager developers would do well to either implement similar systems in their projects This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. It's a challenge because constraint solving is NP-complete, but it works very well in practice. [1]: https://pub.dartlang.org/ https://pub.dartlang.org/ > realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers. The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.

davexunit 11y ago
>This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. Constraint solving is the status quo, and I am strongly opposed to it. Rather than saying "I want foo >= 1.0" in a package manager, you should have the build system test the environment for the features you need. This is what the Autotools and other build systems that have been around for decades do. This way, the project isn't tied to a single package manager and, in the case of free software, greatly eases the burden on package maintainers that want to add your software to their distribution. Functional package management doesn't need constraint solving because every package in the system precisely describes itself: exactly which dependencies are needed for build-time and runtime (which precisely describe themselves, recursively, all the way down to libc), precisely which source code (tarball/directory/whatever + checksum), and the exact build script that turns that source code into a (hopefully bit-reproducible) binary. There's no constraint solving to determine what the dependency tree is, it's already been encoded in the package objects themselves. This is what any robust package manager should enable, but the only two that I know of that do this are Nix and GNU Guix. >The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes. A programming language should not depend on any specific package manager. This is what build systems are for! More and more language communities conflate the two, but this is a huge mistake that we are paying for by making our software nearly impossible to reproduce from source code. [0] Everyone just uses pre-built binaries full of circular dependencies and reproducibility issues because things are so tangled that no one actually knows how to build anything from source anymore. I think the value of language-specific package managers lies in providing an easy way to fetch pure (no native code) modules to faciliate simple code sharing and helping newcomers get bootstrapped quickly. However, for serious software development and deployment they are terrible and we desperately need better tools. I think functional package managers are the tools we need, as they have greatly simplified software building and deployment for me whilst also greatly increasing the reliability of the systems that use them thanks to transactional upgrades and rollbacks. [0] http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html http://www.vitavonni.de/blog/201503/2015031201-the-sad-state...
- nawitus 11y ago
  Language specific package managers seem to actually work in real life and are cross platform, e.g. npm. But I'm not aware of anything for e.g. C++ which would offer the features of npm (mainly installing dependencies with a single command and supporting GNU/Linux, OS X and Windows). Is there a single non-language specific package manager which is cross platform, and supports a large percentage of packages for a given programming language? Note that I'm not saying npm is excellent, but I'm not aware of a single successful language-independent package manager. If there are none, why?
  - gcr 11y ago
    Conda aspires to be exactly that. It attempts to package the entire Python scientific stack, but its general purpose nature lets you package other kinds of software as well (for instance, conda can replace compiling arbitrary C programs from source and installing them into your home folder)
  - Arcsech 11y ago
    The aforementioned GNU Guix and the similar Nix project are exactly that: Cross-platform (mostly, at least Nix requires a unixy layer on Windows) language-independent package managers. One could argue that they aren't "successful" (in that they don't have all the packages you might want), but they are still quite new and are pretty different from traditional package managers. I believe that Nix/Guix are the future, but a future that will take some time to become widespread.
    - Touche 11y ago
      > Cross-platform (mostly, at least Nix requires a unixy layer on Windows) Not good enough, language package managers must work on Windows, with a regular command prompt.
      - davexunit 11y ago
        Windows is hopeless. If Windows is a must, you really cannot do much better than a package manager per language that you use. No binary reproducibility, no transactional management, no provenance, no thanks.

alextgordon 11y ago

When we ship node modules to be embedded into Chocolat.app, we have to strip out all the tests and documentation because otherwise it bloats the download size. And the computer it eventually runs on may not be connected to the internet. The nice thing about node is that it's quite small (3.4MB zipped). We don't have to ship npm with the app, and certainly an OS level package manager would be way too large!

nawitus 11y ago
Each package should strip tests and documentation from the package. Of course, many don't..
- tracker1 11y ago
  True enough... it's not always easy to consider as creating an npm module and building/pushing to NPM is really easy to do.

paulyg 11y ago

This is also what Composer (package manager for PHP) did. They ported openSUSE's libzypp to PHP to solve the dependency management problem. After resolving versions the dependencies are installed in a flat list.