4 ms·

Why is it still possible to do when it’s been done I think 2 other times? How crazy that there’s not like a limit or some kind of basic check to prevent the up

Why is it still possible to do when it’s been done I think 2 other times? How crazy that there’s not like a limit or some kind of basic check to prevent the upload or something? Honestly at this point it seems to be a failure on NPM’s part. People should really direct their anger to the maintainers at this point

0cf8612b2e1e 3y ago
What is the correct mitigation for this? Packages can have large dependency trees, so outside of putting an arbitrarily high upper limit, what is the fix?
- vitiral 3y ago
  I think you named at least one reasonable fix: an upper limit. And it's not really arbitrary. You can count the dependency tree of every package and get the P99, exclude joke packages and get the max of the rest. Set the limit around there with an error message to open a bug if you need more, grandfathering legit old large packages with some kind of admin-level exception. The number of people actually needing more will be incredibly tiny. Service those folks individually to avoid pain.
  - joemi 3y ago
    It's still arbitrary because P99 is an arbitrary percentage. Why not P97 or P90? Or P99.9? And if you have an arbitrary cutoff but make exceptions for _some_ people who are over that cutoff, you're opening yourself up to two things you don't want: more work for yourself to review the special cases, and more complaining from people. Better to have a rule apply to everyone equally.
    - vitiral 3y ago
      The cutoff is more about "this is the number of bug requests for exceptions" rather than some kind of technical limit. When it's a staffing limit the question becomes less arbitrary.
  - hombre_fatal 3y ago
    Your policy wouldn't change anything here. The person made thousands of modules that look like `@everything/sub-chunk-{id}`, each depending on N modules. Those sub-chunk modules, together, depend on all NPM modules. Whatever upper limit you come up with, anyone can do this by setting N to your upper limit. The problem here isn't that you can create modules that depend on other modules. If there's a problem at all, it's that the NPM's yank policy could be improved.
    - vitiral 3y ago
      My suggestion was for N=recursive deps (dependency TREE), so yes it would affect the "everything" package, and likely many of its chunks And I agree about yank! I just think you could implement multiple reasonable solutions here
      - hombre_fatal 3y ago
        It would affect the everything package, but not the issue since the issue is caused by npm's policy + simply creating thousands packages with, say, 5 dependencies that cover all of the module space. That they rolled all of those submodules into one big everything package is just a gimmick, not a source of the problem.
- j-bos 3y ago
  This seems reasonable: https://news.ycombinator.com/item?id=38874889 https://news.ycombinator.com/item?id=38874889

0cf8612b2e1e 3y ago

What is the correct mitigation for this? Packages can have large dependency trees, so outside of putting an arbitrarily high upper limit, what is the fix?

vitiral 3y ago
I think you named at least one reasonable fix: an upper limit. And it's not really arbitrary. You can count the dependency tree of every package and get the P99, exclude joke packages and get the max of the rest. Set the limit around there with an error message to open a bug if you need more, grandfathering legit old large packages with some kind of admin-level exception. The number of people actually needing more will be incredibly tiny. Service those folks individually to avoid pain.
- joemi 3y ago
  It's still arbitrary because P99 is an arbitrary percentage. Why not P97 or P90? Or P99.9? And if you have an arbitrary cutoff but make exceptions for _some_ people who are over that cutoff, you're opening yourself up to two things you don't want: more work for yourself to review the special cases, and more complaining from people. Better to have a rule apply to everyone equally.
  - vitiral 3y ago
    The cutoff is more about "this is the number of bug requests for exceptions" rather than some kind of technical limit. When it's a staffing limit the question becomes less arbitrary.
- hombre_fatal 3y ago
  Your policy wouldn't change anything here. The person made thousands of modules that look like `@everything/sub-chunk-{id}`, each depending on N modules. Those sub-chunk modules, together, depend on all NPM modules. Whatever upper limit you come up with, anyone can do this by setting N to your upper limit. The problem here isn't that you can create modules that depend on other modules. If there's a problem at all, it's that the NPM's yank policy could be improved.
  - vitiral 3y ago
    My suggestion was for N=recursive deps (dependency TREE), so yes it would affect the "everything" package, and likely many of its chunks And I agree about yank! I just think you could implement multiple reasonable solutions here
    - hombre_fatal 3y ago
      It would affect the everything package, but not the issue since the issue is caused by npm's policy + simply creating thousands packages with, say, 5 dependencies that cover all of the module space. That they rolled all of those submodules into one big everything package is just a gimmick, not a source of the problem.
j-bos 3y ago
This seems reasonable: https://news.ycombinator.com/item?id=38874889 https://news.ycombinator.com/item?id=38874889