9 ms·

'everything' blocks devs from removing their own NPM packages

116 points by alexzeitler 3y ago

ano-ther 3y ago
npm’s policy makes sense. But couldn’t it be circumvented by “updating” your package to an empty file or nonsense code?
- baobun 3y ago
  No. If foo@1.2.3 is already published, you can not overwrite foo@1.2.3 - you'd publish foo@any.other.version (typically something at least ^1.2.4) and mark foo@1.2.3 as deprecated (which won't actually make it unavailable like an unpublish would).
droopyEyelids 3y ago
I'd love to see a infographic style explanation website of all the different package managers and their tradeoffs, written to a package-maintainer level of detail. It's such an interesting and important field but largely treated as an annoyance, dismissed by the very people who should be learning about it. Simply having a table of all the fundamental concepts involved and how each package manager handles them would be really interesting. Or even just a list of the fundamental concepts useful for building your mental model of how they work!
orenlindsey 3y ago
The JS ecosystem is so goofy. Stuff like this, leftpad, and dozens more happen. Doesn't happen to literally any other programming language (or maybe I just haven't paid attention to others)
- infotainment 3y ago
  I'm curious if this type of issue is unique to NPM. Does PIP (Python) have this issue?
  - fragmede 3y ago
    PyPi, but yes. https://securitylabs.datadoghq.com/articles/guarddog-identify-malicious-pypi-packages/#:~:text=Malicious%20PyPI%20packages%20have%20enabled,and%20perform%20cryptojacking%20(colourama) https://securitylabs.datadoghq.com/articles/guarddog-identif.... The link also mentions that PHP has been attacked in this way. As has rust. https://blog.rust-lang.org/2022/05/10/malicious-crate-rustdecimal.html https://blog.rust-lang.org/2022/05/10/malicious-crate-rustde...
    - kibwen 3y ago
      Malicious packages and typosquatting are very different from what's being talked about here. The problem in this case is a clash of policies: NPM historically allowed people to delete their packages after they published them, and then after left-pad they realized that was a terrible idea from an ecosystem reliability perspective so they made it so that you can't unpublish packages that are listed as dependencies of other packages, and so someone took this to the logical conclusion and a made a package that was dependent on all other packages. NPM wants to have its cake and eat it too, which is the problem here. The solution is just to say that if you publish a package to NPM you give it the perpetual right to distribute it as-is, and then remove the ability for users to delete their packages at-will.
      - wruza 3y ago
        When authors want to remove a package, realistically there are two main reasons: (a) they don’t need it anymore or (b) there’s a serious issue with it in general, like deprecation or unfitness. Why not just (a) hide it in ui or (b) flag it, explain the reason and give everyone enough time to migrate away. So many issues arise from the fact that our UIs can’t do simple things on lists like folders (e.g. archive) and tags/notes.
      - minot 3y ago
        > NPM wants to have its cake and eat it too, which is the problem here. The solution is just to say that if you publish a package to NPM you give it the perpetual right to distribute it as-is, and then remove the ability for users to delete their packages at-will. Yes, exactly! Kind of unrelated, but I think it is important to remember that the left-pad was also ENTIRELY npm team's fault. You can't just take away a namespace from someone just because some startup like kik comes knocking. Toyota does not have a right to my domain dot tld slash toyota The correct answer would have been npm to tell kik to pound sand. npm has never fixed this grave error. https://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm https://blog.npmjs.org/post/141577284765/kik-left-pad-and-np... > We stand by our package name dispute resolution policy, and the decision to which it led us. npm deserves to die.
- throwitaway1123 3y ago
  Ironically, this happened because of a mitigation put in place to prevent the leftpad disaster from happening again. To recap: the developer of leftpad removed his package, which broke packages that depended on it. To prevent this from happening again NPM decided that you can't unpublish a package that has dependents, which I think is a sensible policy. The result of this is that you can prevent anyone's package from being unpublished by depending on it. This 'everything' package simply took this concept to its extreme conclusion. Personally, I think if you're not sure you want a package to be available publicly in perpetuity, you should maybe consider publishing it to a private registry. Even if NPM did allow developers to unpublish any package, there's nothing stopping me from downloading a tarball of your package as soon as you publish it, and hosting a mirror of it somewhere else.
  - stjohnswarts 3y ago
    Maybe something like this should put up big siren noises that it has so many dependencies. Can -anything- have that many dependencies? seems like a check they could build in and not let it go into the npm packages without a review.
    - charcircuit 3y ago
      So just make a new package per dependency then. And have each version of that package depend on each version of the original package.
  - clipsy 3y ago
    > Even if NPM did allow developers to unpublish any package, there's nothing stopping me from downloading a tarball of your package as soon as you publish it, and hosting a mirror of it somewhere else. As far as I can tell, NPM does not enforce any limitations on the license of published packages; I would not assume you can legally do this unless you have verified the package has been published under a license which permits such actions and which you are certain you understand.
    - clipsy 3y ago
- sgarland 3y ago
  The JS ecosystem is deeply unserious.
- userbinator 3y ago
  It's a combination of low barrier to entry and dogmatic adherence to "not reinventing anything" resulting in quantity over quality and massively deep dependency graphs.
- bob1029 3y ago
  When we say "js ecosystem", npm and node are not what come to my mind in 2024. I don't see these as serious contenders anymore.
dahdum 3y ago
So the “prank”, as called in the article, used the same flaw that allowed someone to do it last January. His apology appears deleted along with the repo. Was there a legitimate purpose or is he just a troll?
- chewmieser 3y ago
  Why is it still possible to do when it’s been done I think 2 other times? How crazy that there’s not like a limit or some kind of basic check to prevent the upload or something? Honestly at this point it seems to be a failure on NPM’s part. People should really direct their anger to the maintainers at this point
  - 0cf8612b2e1e 3y ago
    What is the correct mitigation for this? Packages can have large dependency trees, so outside of putting an arbitrarily high upper limit, what is the fix?
    - vitiral 3y ago
      I think you named at least one reasonable fix: an upper limit. And it's not really arbitrary. You can count the dependency tree of every package and get the P99, exclude joke packages and get the max of the rest. Set the limit around there with an error message to open a bug if you need more, grandfathering legit old large packages with some kind of admin-level exception. The number of people actually needing more will be incredibly tiny. Service those folks individually to avoid pain.
      - joemi 3y ago
        It's still arbitrary because P99 is an arbitrary percentage. Why not P97 or P90? Or P99.9? And if you have an arbitrary cutoff but make exceptions for _some_ people who are over that cutoff, you're opening yourself up to two things you don't want: more work for yourself to review the special cases, and more complaining from people. Better to have a rule apply to everyone equally.
charcircuit 3y ago
I wonder how many bots tried to install it
wilg 3y ago
I'll say it again: why does each language need its own package manager? Why not generalize it to delivering files from various online sources, with a thin layer of glue for module loading that is language-specific.
- krapp 3y ago
  What you're advocating is essentially the way Javascript originally worked, with the web itself serving as the "package manager." but one day someone decided Javascript needed to be a serious business language for serious business, and serious business languages use compilers and toolchains and have a package manager. And so here we are.
  - skybrian 3y ago
    Adding a runtime dependency on some random webserver seems bad? It could be fixed by using reliable CDN's though.
    - krapp 3y ago
      You don't have a runtime dependency if you just download the js to your server and write some script tags. You do have to do things manually and locally but you'd be surprised how robust that can be. Just vendoring scripts and not automating builds would solve like 99% of modern JS problems. The benefit of CDNs used to be that you could cache a library for use on multiple domains but I don't think that works anymore.
      - skybrian 3y ago
        It reminds me of header-only C++ libraries, or downloading jar files manually back before Java had package managers. These don't seem bad for dependency-free libraries, but they won't warn you about incompatible package dependencies.
  - sgarland 3y ago
    "What if we used JS as a backend language?" SMACK Somewhere in an alternate universe, there's a better timeline.
radicality 3y ago
Why is it allowed at all to delete packages? The article states that’s it’s not an issue in Maven since you can’t unpublish anything, only deploy a newer version.
- codetrotter 3y ago
  I like the approach that Rust takes with Cargo and crates.io You can mark a package you’ve published as “yanked”, which will make it so that if someone tries to depend on that version in the future they “can’t” (cargo will tell you it’s yanked). But if it’s in your Cargo.lock file since before, cargo will still fetch it and use it. This way you can mark a broken version of your published package as bad so that fewer people will use that version. While not preventing anyone who was already using it from continuing to use it.
  - c0wb0yc0d3r 3y ago
    Generally, this is how it works with NuGet, too. https://learn.microsoft.com/en-us/nuget/nuget-org/policies/deleting-packages https://learn.microsoft.com/en-us/nuget/nuget-org/policies/d...
- ivanjermakov 3y ago
  Maybe NPM was also designed in 10 days like JavaScript.
aftbit 3y ago
no-one-left-behind[1] was not even a year ago, and yet the link on npm[2] explaining why it was removed is already broken. Classic JS ecosystem. 1: https://github.com/npm/feedback/discussions/858 https://github.com/npm/feedback/discussions/858 2: https://www.npmjs.com/package/no-one-left-behind https://www.npmjs.com/package/no-one-left-behind
- throwitaway1123 3y ago
  I don't think it was ever a link to an explanation. It's just a link to the Github advisories page with a search query for 'no-one-left-behind' attached: www.npmjs.com/advisories?search=no-one-left-behind. It's the equivalent of linking to a Google search. Given the number of bots uploading malicious packages, I wouldn't be surprised if this is mostly all automated, and there's no manpower available to dedicate to descriptions of each shady package uploaded to the world's largest package repository. If this were a vulnerability in the infrastructure of NPM itself, then I would definitely expect a full writeup with a working link.
  - aftbit 3y ago
    Okay, well... then it is broken in THREE ways. First, the link no longer leads to the search, as the redirect seems to strip the parameters. Second, searching the github for "no-one-left-behind" gives no results. Third, this _is_ a vulnerability of the NPM ecosystem, as shown by the fact that this happened _AGAIN_ this week.
    - throwitaway1123 3y ago
      > First, the link no longer leads to the search, as the redirect seems to strip the parameters. Github acquired NPM, and now they're trying to integrate NPM with all of Github's products, so NPM advisories now link to the Github advisories page, which is probably why the search query is being stripped. This is definitely not ideal, but I would probably characterize this as corporate ineptitude. Classic case of a company acquiring a smaller company and not properly handling the integration. > Second, searching the github for "no-one-left-behind" gives no results. Like I said, there are bots publishing malicious packages faster than human reviewers can even describe them, so I don't expect a full write up for every prank package. Are you really upset that you had to Google the package instead of following a link in a package's description? Really? Would you instantly mistrust any organization with a link that lead to a 404 page? > Third, this _is_ a vulnerability of the NPM ecosystem, as shown by the fact that this happened _AGAIN_ this week. I would hardly characterize this as a vulnerability of the NPM ecosystem. I'm not even sure if this would warrant its own CVE. You act as if the package was installing bitcoin miners on developer machines. Like I said, if this were a vulnerability in the infrastructure of NPM itself (say for instance it managed to hack the servers hosting NPM), then I would definitely expect a full write up. All it's doing is preventing people from unpublishing packages. It was honestly a curtesy of NPM to even take it down. If it was up to me I might make the whole registry immutable. Another commenter said it best: "In any case this is a mild inconvience at most. Anyone who gets upset over this needs to go touch grass."[1] [1] https://news.ycombinator.com/item?id=38883067 https://news.ycombinator.com/item?id=38883067
javawizard 3y ago
Fool me once, shame on you. Fool me twice, shame on me. I get that this royally inconvenienced a ton of people, but come on: the fact that this can happen in the first place, along with the fact that the expectation for most users is that it can't, is a clear failing on NPM's part. I don't know what the solution is - maybe packages should be permanent after 24 hours instead of after they're depended upon, maybe there's a different solution - but it's disheartening to see the vitriol with which folks went after PatrickJS in the screenshotted comments in TFA.
- wruza 3y ago
  That’s what you get after making someone’s life harder. I’m all for positiveness, but what did PatrickJS expect? It’s understandable when someone has a monetary incentive, but this is basically a (light) hooliganism. Protecting from all sorts of jokes makes life harder too. Corner cases are hard. Why can’t we just respect each other and have these few nice things that are left. In the next episode we’ll be unable to publish more than three packages a day, have to request dependency quotas, investigate ip blocking issues, solve captcha for every basic operation and try to weed through support rounds to unlock the org that was locked due to suspicious activity. That said, props for them not hiding in a shadow watching things happen.
  - chmod775 3y ago
    > I’m all for positiveness, but what did PatrickJS expect? It’s understandable when someone has a monetary incentive, but this is basically a (light) hooliganism. You're making the assumption that they were aware of the complications their experiment would cause beforehand. The fact that they weren't "hiding in the shadows" indicates they had no ill intent. Also if they didn't do this, someone else eventually would (possible with ill intent). In any case this is a mild inconvience at most. Anyone who gets upset over this needs to go touch grass.
bitwize 3y ago
I'm surprised the next three sublevels weren't named "im-invisible", "an-eraser-of-love", and "why-dontcha-call-me-i-feel-like-flyin-in-two".
SPBS 3y ago
The article tries to make it sound more serious than it actually is. So you can't unpublish packages on NPM now. So what? Why are you unpublishing it in the first place? Just leave it alone and slap a deprecation notice on it.
- matheusmoreira 3y ago
  Because I'm human and sometimes I make idiotic mistakes and I don't want that embarrassing stuff out there where everyone can see. Same reason I sometimes force push to my repositories: I realized I made a mistake immediately after publishing and would like to correct it as quickly as possible. Some time ago I ran head first into an npm bug when I tried to symlink the README file which resulted in the package getting published without a README file because apparently someone somewhere is "opinionated" about symlinks. https://github.com/npm/cli/issues/6746 https://github.com/npm/cli/issues/6746 That was embarrassing and annoying. So I unpublished it. Then I discovered I couldn't replace the bad package, I had to create a new version of the package. So I did and then I discovered they had slapped me with a stupid 24 hour count down on top of all that during which I could not actually push the corrected version. I hate this thing so much, you have no idea.
- richbell 3y ago
  > Why are you unpublishing it in the first place? Just leave it alone and slap a deprecation notice on it. This doesn't work if you've accidentally published sensitive information or files. Though I agree that deleting packages is generally not necessary.
  - joshstrange 3y ago
    OTOH if you’ve published sensitive info the cat is out of the bag. Deleting the version is not the correct response, rotating creds is the only viable answer. I’m sure there are bots watching new versions posted to NPM, scanning them for things like AWS access keys or crypto wallet creds. I’d bet that if you publish a package with AWS creds you will have crypto miners launched on your account within the hour if not within 5 minutes (probably all automated).
userbinator 3y ago
As someone who frequently chases dead links and tries to piece together things from archives: Good. We need things to be more permanent. Storage is cheap, history is priceless.
- internetter 3y ago
  The ironic thing is after all this GitHub accused us of harassment and deleted the GitHub organization (that contained numerous instances of us being harassed) I haven’t read TFA, but judging from the domain it missed the whole point. We weren’t intending to break anything, we just wanted to install everything. Combined storage size of package manifests were 82mb
goodmonson 3y ago
I wonder what other pathological packages could be devised. Like, a package that only includes packages that are not included anywhere else. It could be called “barber”.
stephenr 3y ago
I've said it before and I'll say it again (and this applies to other scripting language package managers too, e.g. composer for php): while "this public package was unpublished" is probably something a package repo shouldn't support, the actual solution is pretty simple. Vendor your dependencies. By which I mean: commit the dependencies you rely on, to your VCS. It's really that simple. Yes, this probably requires some different usage or default behaviour from package managers to be "efficient" (i.e. you probably don't need to include all the tests, docs etc for each dependency in your repo). If you're following "best practices" for every library package manager I've seen (to commit their respective "lock" file, thereby fixing the used version of each dependency for subsequent installs), the end result is the same in terms of what version of dependency is available when the source is checked out - but now you have (a) zero risk of dependencies being pulled; (b) the ability to do comprehensive change reviews when versions are bumped; (c) likely faster updates for anyone pulling changes subsequent to a dependency change; (d) zero risk of 'install dependencies' related issues at deployment time.
- shitlord 3y ago
  This only works to a point. When you install dependencies with a tool like npm install, you're installing the dependencies meant for your computer platform. You can't just copy binaries between computers and expect them to work. Even with the same OS you could be missing shared libraries.
  - stephenr 3y ago
    Can you give an example of this phenomenon, e.g. a package that one would install using `npm` which includes binaries? I wasn't even aware this was a thing people do.
    - foldr 3y ago
      Loads of node packages depend on native code in one form or another. E.g. https://www.npmjs.com/package/sqlite3 https://www.npmjs.com/package/sqlite3
    - 3y ago
TheAceOfHearts 3y ago
I'm most disappointed that not a single person has asked the most pertinent question: what's the total size of node_modules after successfully installing the everything package?
- ht85 3y ago
  Just tried installing it, fails as some of the dependencies have been made private :(
hiccuphippo 3y ago
Now someone needs to make a package that conflicts with every other package and the circle will be closed.
20after4 3y ago
They should have named it the everything bagel.
gempir 3y ago
Unpopular take on NPM: NPM is a very good package manager registry. It's just extremely popular because javascript is extremely popular and the entry barrier is tiny compared to other languages and registries. These issues would pop up in any package manager, no matter the language, you just need enough users and usecases.
- stonogo 3y ago
  When did this happen in CPAN?
- sebazzz 3y ago
  No, NPM has never been thought out well. Many of the improvements in npm over the years were all either (1) due to accidents that many other package managers like NuGet already prevent by default or (2) features that were already implemented in Yarn 1.x.

116 points by alexzeitler 3y ago

ano-ther 3y ago

npm’s policy makes sense. But couldn’t it be circumvented by “updating” your package to an empty file or nonsense code?

baobun 3y ago
No. If foo@1.2.3 is already published, you can not overwrite foo@1.2.3 - you'd publish foo@any.other.version (typically something at least ^1.2.4) and mark foo@1.2.3 as deprecated (which won't actually make it unavailable like an unpublish would).

droopyEyelids 3y ago

I'd love to see a infographic style explanation website of all the different package managers and their tradeoffs, written to a package-maintainer level of detail. It's such an interesting and important field but largely treated as an annoyance, dismissed by the very people who should be learning about it. Simply having a table of all the fundamental concepts involved and how each package manager handles them would be really interesting. Or even just a list of the fundamental concepts useful for building your mental model of how they work!

orenlindsey 3y ago

The JS ecosystem is so goofy. Stuff like this, leftpad, and dozens more happen. Doesn't happen to literally any other programming language (or maybe I just haven't paid attention to others)

infotainment 3y ago
I'm curious if this type of issue is unique to NPM. Does PIP (Python) have this issue?
- fragmede 3y ago
  PyPi, but yes. https://securitylabs.datadoghq.com/articles/guarddog-identify-malicious-pypi-packages/#:~:text=Malicious%20PyPI%20packages%20have%20enabled,and%20perform%20cryptojacking%20(colourama) https://securitylabs.datadoghq.com/articles/guarddog-identif.... The link also mentions that PHP has been attacked in this way. As has rust. https://blog.rust-lang.org/2022/05/10/malicious-crate-rustdecimal.html https://blog.rust-lang.org/2022/05/10/malicious-crate-rustde...
  - kibwen 3y ago
    Malicious packages and typosquatting are very different from what's being talked about here. The problem in this case is a clash of policies: NPM historically allowed people to delete their packages after they published them, and then after left-pad they realized that was a terrible idea from an ecosystem reliability perspective so they made it so that you can't unpublish packages that are listed as dependencies of other packages, and so someone took this to the logical conclusion and a made a package that was dependent on all other packages. NPM wants to have its cake and eat it too, which is the problem here. The solution is just to say that if you publish a package to NPM you give it the perpetual right to distribute it as-is, and then remove the ability for users to delete their packages at-will.
    - wruza 3y ago
      When authors want to remove a package, realistically there are two main reasons: (a) they don’t need it anymore or (b) there’s a serious issue with it in general, like deprecation or unfitness. Why not just (a) hide it in ui or (b) flag it, explain the reason and give everyone enough time to migrate away. So many issues arise from the fact that our UIs can’t do simple things on lists like folders (e.g. archive) and tags/notes.
    - minot 3y ago
      > NPM wants to have its cake and eat it too, which is the problem here. The solution is just to say that if you publish a package to NPM you give it the perpetual right to distribute it as-is, and then remove the ability for users to delete their packages at-will. Yes, exactly! Kind of unrelated, but I think it is important to remember that the left-pad was also ENTIRELY npm team's fault. You can't just take away a namespace from someone just because some startup like kik comes knocking. Toyota does not have a right to my domain dot tld slash toyota The correct answer would have been npm to tell kik to pound sand. npm has never fixed this grave error. https://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm https://blog.npmjs.org/post/141577284765/kik-left-pad-and-np... > We stand by our package name dispute resolution policy, and the decision to which it led us. npm deserves to die.
throwitaway1123 3y ago
Ironically, this happened because of a mitigation put in place to prevent the leftpad disaster from happening again. To recap: the developer of leftpad removed his package, which broke packages that depended on it. To prevent this from happening again NPM decided that you can't unpublish a package that has dependents, which I think is a sensible policy. The result of this is that you can prevent anyone's package from being unpublished by depending on it. This 'everything' package simply took this concept to its extreme conclusion. Personally, I think if you're not sure you want a package to be available publicly in perpetuity, you should maybe consider publishing it to a private registry. Even if NPM did allow developers to unpublish any package, there's nothing stopping me from downloading a tarball of your package as soon as you publish it, and hosting a mirror of it somewhere else.
- stjohnswarts 3y ago
  Maybe something like this should put up big siren noises that it has so many dependencies. Can -anything- have that many dependencies? seems like a check they could build in and not let it go into the npm packages without a review.
  - charcircuit 3y ago
    So just make a new package per dependency then. And have each version of that package depend on each version of the original package.
- clipsy 3y ago
  > Even if NPM did allow developers to unpublish any package, there's nothing stopping me from downloading a tarball of your package as soon as you publish it, and hosting a mirror of it somewhere else. As far as I can tell, NPM does not enforce any limitations on the license of published packages; I would not assume you can legally do this unless you have verified the package has been published under a license which permits such actions and which you are certain you understand.
  - clipsy 3y ago
sgarland 3y ago
The JS ecosystem is deeply unserious.
userbinator 3y ago
It's a combination of low barrier to entry and dogmatic adherence to "not reinventing anything" resulting in quantity over quality and massively deep dependency graphs.
bob1029 3y ago
When we say "js ecosystem", npm and node are not what come to my mind in 2024. I don't see these as serious contenders anymore.

dahdum 3y ago

So the “prank”, as called in the article, used the same flaw that allowed someone to do it last January. His apology appears deleted along with the repo. Was there a legitimate purpose or is he just a troll?

chewmieser 3y ago
Why is it still possible to do when it’s been done I think 2 other times? How crazy that there’s not like a limit or some kind of basic check to prevent the upload or something? Honestly at this point it seems to be a failure on NPM’s part. People should really direct their anger to the maintainers at this point
- 0cf8612b2e1e 3y ago
  What is the correct mitigation for this? Packages can have large dependency trees, so outside of putting an arbitrarily high upper limit, what is the fix?
  - vitiral 3y ago
    I think you named at least one reasonable fix: an upper limit. And it's not really arbitrary. You can count the dependency tree of every package and get the P99, exclude joke packages and get the max of the rest. Set the limit around there with an error message to open a bug if you need more, grandfathering legit old large packages with some kind of admin-level exception. The number of people actually needing more will be incredibly tiny. Service those folks individually to avoid pain.
    - joemi 3y ago
      It's still arbitrary because P99 is an arbitrary percentage. Why not P97 or P90? Or P99.9? And if you have an arbitrary cutoff but make exceptions for _some_ people who are over that cutoff, you're opening yourself up to two things you don't want: more work for yourself to review the special cases, and more complaining from people. Better to have a rule apply to everyone equally.

charcircuit 3y ago

I wonder how many bots tried to install it

wilg 3y ago

I'll say it again: why does each language need its own package manager? Why not generalize it to delivering files from various online sources, with a thin layer of glue for module loading that is language-specific.

krapp 3y ago
What you're advocating is essentially the way Javascript originally worked, with the web itself serving as the "package manager." but one day someone decided Javascript needed to be a serious business language for serious business, and serious business languages use compilers and toolchains and have a package manager. And so here we are.
- skybrian 3y ago
  Adding a runtime dependency on some random webserver seems bad? It could be fixed by using reliable CDN's though.
  - krapp 3y ago
    You don't have a runtime dependency if you just download the js to your server and write some script tags. You do have to do things manually and locally but you'd be surprised how robust that can be. Just vendoring scripts and not automating builds would solve like 99% of modern JS problems. The benefit of CDNs used to be that you could cache a library for use on multiple domains but I don't think that works anymore.
    - skybrian 3y ago
      It reminds me of header-only C++ libraries, or downloading jar files manually back before Java had package managers. These don't seem bad for dependency-free libraries, but they won't warn you about incompatible package dependencies.
- sgarland 3y ago
  "What if we used JS as a backend language?" SMACK Somewhere in an alternate universe, there's a better timeline.

radicality 3y ago

Why is it allowed at all to delete packages? The article states that’s it’s not an issue in Maven since you can’t unpublish anything, only deploy a newer version.

codetrotter 3y ago
I like the approach that Rust takes with Cargo and crates.io You can mark a package you’ve published as “yanked”, which will make it so that if someone tries to depend on that version in the future they “can’t” (cargo will tell you it’s yanked). But if it’s in your Cargo.lock file since before, cargo will still fetch it and use it. This way you can mark a broken version of your published package as bad so that fewer people will use that version. While not preventing anyone who was already using it from continuing to use it.
- c0wb0yc0d3r 3y ago
  Generally, this is how it works with NuGet, too. https://learn.microsoft.com/en-us/nuget/nuget-org/policies/deleting-packages https://learn.microsoft.com/en-us/nuget/nuget-org/policies/d...
ivanjermakov 3y ago
Maybe NPM was also designed in 10 days like JavaScript.

aftbit 3y ago

no-one-left-behind[1] was not even a year ago, and yet the link on npm[2] explaining why it was removed is already broken. Classic JS ecosystem. 1: https://github.com/npm/feedback/discussions/858 https://github.com/npm/feedback/discussions/858 2: https://www.npmjs.com/package/no-one-left-behind https://www.npmjs.com/package/no-one-left-behind

throwitaway1123 3y ago
I don't think it was ever a link to an explanation. It's just a link to the Github advisories page with a search query for 'no-one-left-behind' attached: www.npmjs.com/advisories?search=no-one-left-behind. It's the equivalent of linking to a Google search. Given the number of bots uploading malicious packages, I wouldn't be surprised if this is mostly all automated, and there's no manpower available to dedicate to descriptions of each shady package uploaded to the world's largest package repository. If this were a vulnerability in the infrastructure of NPM itself, then I would definitely expect a full writeup with a working link.
- aftbit 3y ago
  Okay, well... then it is broken in THREE ways. First, the link no longer leads to the search, as the redirect seems to strip the parameters. Second, searching the github for "no-one-left-behind" gives no results. Third, this _is_ a vulnerability of the NPM ecosystem, as shown by the fact that this happened _AGAIN_ this week.
  - throwitaway1123 3y ago
    > First, the link no longer leads to the search, as the redirect seems to strip the parameters. Github acquired NPM, and now they're trying to integrate NPM with all of Github's products, so NPM advisories now link to the Github advisories page, which is probably why the search query is being stripped. This is definitely not ideal, but I would probably characterize this as corporate ineptitude. Classic case of a company acquiring a smaller company and not properly handling the integration. > Second, searching the github for "no-one-left-behind" gives no results. Like I said, there are bots publishing malicious packages faster than human reviewers can even describe them, so I don't expect a full write up for every prank package. Are you really upset that you had to Google the package instead of following a link in a package's description? Really? Would you instantly mistrust any organization with a link that lead to a 404 page? > Third, this _is_ a vulnerability of the NPM ecosystem, as shown by the fact that this happened _AGAIN_ this week. I would hardly characterize this as a vulnerability of the NPM ecosystem. I'm not even sure if this would warrant its own CVE. You act as if the package was installing bitcoin miners on developer machines. Like I said, if this were a vulnerability in the infrastructure of NPM itself (say for instance it managed to hack the servers hosting NPM), then I would definitely expect a full write up. All it's doing is preventing people from unpublishing packages. It was honestly a curtesy of NPM to even take it down. If it was up to me I might make the whole registry immutable. Another commenter said it best: "In any case this is a mild inconvience at most. Anyone who gets upset over this needs to go touch grass."[1] [1] https://news.ycombinator.com/item?id=38883067 https://news.ycombinator.com/item?id=38883067

javawizard 3y ago

Fool me once, shame on you. Fool me twice, shame on me. I get that this royally inconvenienced a ton of people, but come on: the fact that this can happen in the first place, along with the fact that the expectation for most users is that it can't, is a clear failing on NPM's part. I don't know what the solution is - maybe packages should be permanent after 24 hours instead of after they're depended upon, maybe there's a different solution - but it's disheartening to see the vitriol with which folks went after PatrickJS in the screenshotted comments in TFA.

wruza 3y ago
That’s what you get after making someone’s life harder. I’m all for positiveness, but what did PatrickJS expect? It’s understandable when someone has a monetary incentive, but this is basically a (light) hooliganism. Protecting from all sorts of jokes makes life harder too. Corner cases are hard. Why can’t we just respect each other and have these few nice things that are left. In the next episode we’ll be unable to publish more than three packages a day, have to request dependency quotas, investigate ip blocking issues, solve captcha for every basic operation and try to weed through support rounds to unlock the org that was locked due to suspicious activity. That said, props for them not hiding in a shadow watching things happen.
- chmod775 3y ago
  > I’m all for positiveness, but what did PatrickJS expect? It’s understandable when someone has a monetary incentive, but this is basically a (light) hooliganism. You're making the assumption that they were aware of the complications their experiment would cause beforehand. The fact that they weren't "hiding in the shadows" indicates they had no ill intent. Also if they didn't do this, someone else eventually would (possible with ill intent). In any case this is a mild inconvience at most. Anyone who gets upset over this needs to go touch grass.

bitwize 3y ago

I'm surprised the next three sublevels weren't named "im-invisible", "an-eraser-of-love", and "why-dontcha-call-me-i-feel-like-flyin-in-two".

SPBS 3y ago

The article tries to make it sound more serious than it actually is. So you can't unpublish packages on NPM now. So what? Why are you unpublishing it in the first place? Just leave it alone and slap a deprecation notice on it.

matheusmoreira 3y ago
Because I'm human and sometimes I make idiotic mistakes and I don't want that embarrassing stuff out there where everyone can see. Same reason I sometimes force push to my repositories: I realized I made a mistake immediately after publishing and would like to correct it as quickly as possible. Some time ago I ran head first into an npm bug when I tried to symlink the README file which resulted in the package getting published without a README file because apparently someone somewhere is "opinionated" about symlinks. https://github.com/npm/cli/issues/6746 https://github.com/npm/cli/issues/6746 That was embarrassing and annoying. So I unpublished it. Then I discovered I couldn't replace the bad package, I had to create a new version of the package. So I did and then I discovered they had slapped me with a stupid 24 hour count down on top of all that during which I could not actually push the corrected version. I hate this thing so much, you have no idea.
richbell 3y ago
> Why are you unpublishing it in the first place? Just leave it alone and slap a deprecation notice on it. This doesn't work if you've accidentally published sensitive information or files. Though I agree that deleting packages is generally not necessary.
- joshstrange 3y ago
  OTOH if you’ve published sensitive info the cat is out of the bag. Deleting the version is not the correct response, rotating creds is the only viable answer. I’m sure there are bots watching new versions posted to NPM, scanning them for things like AWS access keys or crypto wallet creds. I’d bet that if you publish a package with AWS creds you will have crypto miners launched on your account within the hour if not within 5 minutes (probably all automated).

userbinator 3y ago

As someone who frequently chases dead links and tries to piece together things from archives: Good. We need things to be more permanent. Storage is cheap, history is priceless.

internetter 3y ago
The ironic thing is after all this GitHub accused us of harassment and deleted the GitHub organization (that contained numerous instances of us being harassed) I haven’t read TFA, but judging from the domain it missed the whole point. We weren’t intending to break anything, we just wanted to install everything. Combined storage size of package manifests were 82mb

goodmonson 3y ago

I wonder what other pathological packages could be devised. Like, a package that only includes packages that are not included anywhere else. It could be called “barber”.

stephenr 3y ago

I've said it before and I'll say it again (and this applies to other scripting language package managers too, e.g. composer for php): while "this public package was unpublished" is probably something a package repo shouldn't support, the actual solution is pretty simple. Vendor your dependencies. By which I mean: commit the dependencies you rely on, to your VCS. It's really that simple. Yes, this probably requires some different usage or default behaviour from package managers to be "efficient" (i.e. you probably don't need to include all the tests, docs etc for each dependency in your repo). If you're following "best practices" for every library package manager I've seen (to commit their respective "lock" file, thereby fixing the used version of each dependency for subsequent installs), the end result is the same in terms of what version of dependency is available when the source is checked out - but now you have (a) zero risk of dependencies being pulled; (b) the ability to do comprehensive change reviews when versions are bumped; (c) likely faster updates for anyone pulling changes subsequent to a dependency change; (d) zero risk of 'install dependencies' related issues at deployment time.

shitlord 3y ago
This only works to a point. When you install dependencies with a tool like npm install, you're installing the dependencies meant for your computer platform. You can't just copy binaries between computers and expect them to work. Even with the same OS you could be missing shared libraries.
- stephenr 3y ago
  Can you give an example of this phenomenon, e.g. a package that one would install using `npm` which includes binaries? I wasn't even aware this was a thing people do.
  - foldr 3y ago
    Loads of node packages depend on native code in one form or another. E.g. https://www.npmjs.com/package/sqlite3 https://www.npmjs.com/package/sqlite3
  - 3y ago

TheAceOfHearts 3y ago

I'm most disappointed that not a single person has asked the most pertinent question: what's the total size of node_modules after successfully installing the everything package?

ht85 3y ago
Just tried installing it, fails as some of the dependencies have been made private :(

hiccuphippo 3y ago

Now someone needs to make a package that conflicts with every other package and the circle will be closed.

20after4 3y ago

They should have named it the everything bagel.

gempir 3y ago

Unpopular take on NPM: NPM is a very good package manager registry. It's just extremely popular because javascript is extremely popular and the entry barrier is tiny compared to other languages and registries. These issues would pop up in any package manager, no matter the language, you just need enough users and usecases.

stonogo 3y ago
When did this happen in CPAN?
sebazzz 3y ago
No, NPM has never been thought out well. Many of the improvements in npm over the years were all either (1) due to accidents that many other package managers like NuGet already prevent by default or (2) features that were already implemented in Yarn 1.x.