3 ms·

It's because having an actual shrinkwrap file in a library introduces a huge number of conflicts. If library A has 20 transitive dependencies, and library B has

It's because having an actual shrinkwrap file in a library introduces a huge number of conflicts. If library A has 20 transitive dependencies, and library B has the same 20 transitive dependencies, deduplication based on ranges can leave you with only 20 transitive dependencies. If both libraries have shrinkwrap files in them, you can end up with 40 transitive dependencies, even if those are nominally compatible with each other. When you consider that even with the current behavior projects end up with thousands of transitives, you're looking at adding literally thousands or 10s of thousands of duplicate dependencies to a project. That's not even considering that some duplicate dependencies will just break things if they show up multiple times in a project (older versions of react, for example).

freeqaz 5y ago
Ah, that totally makes sense. I know that's what "peerDependencies" is used for by many projects. For example, `react-dom` has a peerDependency on `react`. This can either be pinned to an exact version (react-dom@1.1.1 must be used with react@1.1.1) or it can require a range (react-dom@1.1.1 may be used with react@^1.0.0 (any version 1.0.0 and up)). That seems tricky with the shrinkwrap, as you describe above, because it might create additional versions everywhere instead of trying to collapse them. (If you have 3 versions of `react` in peerDependencies, but each package had a npm-shrinkwrap.json, then you'd end up with 3 different versions of `react` installed.) Would the burden then fall on your package manager to resolve this, ie npm or yarn? If it sees 3 different shrinkwraps (1.1.1, 1.1.2, 1.1.3) but they all have compatible peerDependencies (>=1.1.1, >=1.1.2, >=1.1.3) it could then pick the version that is compatible with all of the peerDependencies but that also exists in a shrinkwrap (so 1.1.3). That would protect you from this "colors" scenario of somebody publishing `1.1.4` with malicious changes because, unless somebody were to shrinkwrap it, then it wouldn't get picked up by default. (In contrast to the current semver, which likely would pick up 1.1.4 upon a fresh `npm install`). The inverse side of this is highlighted by another comment on this post: That semver makes absorbing security updates an easier process in the event of a log4j-style vulnerability. There is always a tradeoff, for sure!
er4hn 5y ago
It is worth noting that this is not an impossible problem. I'm not a js expert, and it seems that js loves imports, but Nix solves exactly this problem on Linux. You have, for each end binary, a set of dependencies. If they are exactly the same, they can be shared among programs, otherwise you would have different versions of the dependency, all the way down, per program. It can be bloaty, and you have to manually look at and test packages if you want to get everyone on the same set of dependencies, but you end up with reproducible environments.
- tomberek 5y ago
  This is probably the only sane way to proceed when our software has dependency chains more than a few levels deep. Establish mechanisms to try to prevent bloat, but otherwise make it possible upgrade independently and make it starkly apparent if the is duplication.
- V99 5y ago
  That is exactly how npm works. Except instead of one at least somewhat-aligned group of maintainers of a Linux distro trying to keep things under control, you have a much larger set of packages individually maintained (or abandoned) by totally independent people. So practically nothing will ever exactly match, which leaves you right back at many slightly-different copies of hundreds or thousands of dependencies.

freeqaz 5y ago

Ah, that totally makes sense. I know that's what "peerDependencies" is used for by many projects. For example, `react-dom` has a peerDependency on `react`. This can either be pinned to an exact version (react-dom@1.1.1 must be used with react@1.1.1) or it can require a range (react-dom@1.1.1 may be used with react@^1.0.0 (any version 1.0.0 and up)). That seems tricky with the shrinkwrap, as you describe above, because it might create additional versions everywhere instead of trying to collapse them. (If you have 3 versions of `react` in peerDependencies, but each package had a npm-shrinkwrap.json, then you'd end up with 3 different versions of `react` installed.) Would the burden then fall on your package manager to resolve this, ie npm or yarn? If it sees 3 different shrinkwraps (1.1.1, 1.1.2, 1.1.3) but they all have compatible peerDependencies (>=1.1.1, >=1.1.2, >=1.1.3) it could then pick the version that is compatible with all of the peerDependencies but that also exists in a shrinkwrap (so 1.1.3). That would protect you from this "colors" scenario of somebody publishing `1.1.4` with malicious changes because, unless somebody were to shrinkwrap it, then it wouldn't get picked up by default. (In contrast to the current semver, which likely would pick up 1.1.4 upon a fresh `npm install`). The inverse side of this is highlighted by another comment on this post: That semver makes absorbing security updates an easier process in the event of a log4j-style vulnerability. There is always a tradeoff, for sure!

er4hn 5y ago

It is worth noting that this is not an impossible problem. I'm not a js expert, and it seems that js loves imports, but Nix solves exactly this problem on Linux. You have, for each end binary, a set of dependencies. If they are exactly the same, they can be shared among programs, otherwise you would have different versions of the dependency, all the way down, per program. It can be bloaty, and you have to manually look at and test packages if you want to get everyone on the same set of dependencies, but you end up with reproducible environments.

tomberek 5y ago
This is probably the only sane way to proceed when our software has dependency chains more than a few levels deep. Establish mechanisms to try to prevent bloat, but otherwise make it possible upgrade independently and make it starkly apparent if the is duplication.
V99 5y ago
That is exactly how npm works. Except instead of one at least somewhat-aligned group of maintainers of a Linux distro trying to keep things under control, you have a much larger set of packages individually maintained (or abandoned) by totally independent people. So practically nothing will ever exactly match, which leaves you right back at many slightly-different copies of hundreds or thousands of dependencies.