12 ms·

Alert: NPM modules hijacked

238 points by vdfs 11y ago

SlashmanX 11y ago
It's insane that NPM allows anybody to just take over a module name if the original is unpublished, without even a warning to users.
- henrikfr 11y ago
  Insane, yes, but I guess it's just never really been that much of a problem until now. Hopefully something good will come out of this..
  - smt88 11y ago
    > it's just never really been that much of a problem until now That's a reasonable excuse in the 90s, before automatic updates over HTTP were common. Our industry now has decades of experience securing HTTP updates and package managers, with various Linux solutions demonstrating good practices.
  - CorpOverreach 11y ago
    You would think this is part of security 101 for these things... And people wonder why big enterprises are scared of touching open source stuff.
    - nyan4 11y ago
      > And people wonder why big enterprises are scared of touching open source stuff. some open source stuff. Most enterprises dig distributions, especially with LTS.
    - cyphar 11y ago
      Many big enterprises run on free software. Just because this particular free software project is a shit-show doesn't make all free software projects this bad.
  - gst 11y ago
    Something similar is (or at least was) true for Github: I changed my username a couple of years ago and I recently found out that another user is now using my original username. Less of an issue due to the much longer timeframe, but I think not allowing username re-use would be a safer choice.
  - FranOntanaya 11y ago
    Module repositories have been a thing for decades. Even a platform as crusty-old as WordPress doesn't allow repository take-overs for plugins. Surely someone had some experience outside npm to think about this.
- na85 11y ago
  Insane, yes, but not unexpected. This is the sort of amateur behaviour that is emblematic of the entire nodejs community.
  - akerro 11y ago
    That's what happens when a language and framework have very low entry level, more and more not responsible people gather in one place. Yes, they make it popular and trendy, but they're dangerous for themselves and others.
    - petetnt 11y ago
      Amazingly shameful cheap shots and mudslinging against hundreds of people from both you and @na85. Bravo.
      - stonogo 11y ago
        Those people deserve criticism, because they're making bad decisions.
        sangnoir 11y ago
        and by 'Those people', you mean the entire NodeJS community? > This is the sort of amateur behaviour that is emblematic of the entire nodejs community. This is pure mudslinging
        akerro 11y ago
        No, I'm sure he didn't meant whole community. There was a reason why njs became so popular, and there are great people behind it, but as everything that's easy to start with, it attracts people with no skills, no experience. When learning C one of the first things you learn is how operating systems work, what memory leaks are, why they are dangerous, C, C++ have higher entry level, people who stay longer with C/C++ have very good experience, they know more about security, risks, standards, systems... When you have low entry lever, you don't even bother reading on wiki what XSS is, what .ssh folder is. We just learnt the hard way that package manager for NJS is crap, just to compare it, take a look at crates of a language that has high entry level: https://www.reddit.com/r/rust/comments/4bm3rk/how_would_cratesio_react_in_a_case_similar_to_the/ https://www.reddit.com/r/rust/comments/4bm3rk/how_would_crat...
- viperscape 11y ago
  https://twitter.com/_scape/status/700453739182817281 https://twitter.com/_scape/status/700453739182817281 My semi relevant tweet, it's just asking for it with that warning
  - irl_zebra 11y ago
    Why not just post the tweet here instead of spamming your Twitter feed?
    - viperscape 11y ago
      That's a perma link to the direct tweet... why paste in the tweet when I can just link it...
      - prodigal_erik 11y ago
        HN would be unusable if everyone stopped writing comments and just pasted links to an ephemeral third-party service.
      - aw3c2 11y ago
        http://pastebin.com/XkPwwkUs http://pastebin.com/XkPwwkUs
      - douche 11y ago
        Bravo, you've illustrated the whole problem
- drdaeman 11y ago
  Link rot and domain squatting are a well-known decades-old issues. NPM identifiers can be considered as just another weird form of pseudo-URIs and aren't special here. It is insane that modules don't have signatures (that are actually verified, of course). Because npm can feed you basically anything. It's not a problem that something else gets published under the old address. It's perfectly natural. The real problem is the trust model - that new content's accepted without even warnings.
- choward 11y ago
  It's insane that NPM allows anybody to un-publish modules to begin with. Even if I have a dependency locked down in my NPM shrinkwrap file, it can change underneath me? That is pretty absurd and gives me zero confidence in my packages. It means I MUST commit them to source control or risk having my project completely broken some day. I thought for sure that since NPM removed the ability to re-publish the same version of a package with different content that they also wouldn't let you remove versions of a package. It also means you should never user the "^" version specifier or risk downloading some completely different project. They do have a "warning" at https://docs.npmjs.com/cli/unpublish https://docs.npmjs.com/cli/unpublish: > WARNING > > It is generally considered bad behavior to remove versions of a library that others are depending on! Seriously, what good does that do? Nobody takes warnings seriously.
  - Wintamute 11y ago
    A specific version of a package is immutable on npm. So if you depend on exactly foo@1.2.3 in your shrinkwrap you'll always get the same code. In other words, even if the package is unpublished, and picked up by a new user they can't re-publish new code to 1.2.3. If you depend on a version range like ^1.2.3 or ~1.2.3 its a different story, of course. Moral of the story is imo always pin to exact versions and use shrinkwrap for production apps. EDIT: This may not be true .. see below.
    - choward 11y ago
      > A specific version of a package is immutable on NPM. That's not true. I thought it was but it's not. That's the terrible part. To demonstrate with one of the packages that was removed, run: $ npm info andthen You'll see that versions 0.0.1 and 0.0.2 were published at one point. However, for "versions" it only mentions 2.0.0. And of course, if you run: $ npm install andthen@0.0.2 It blows up in your face.
      - Wintamute 11y ago
        :o
pluma 11y ago
Overly dramatic. The content is not suspicious at all, it's clearly a script to generate the package.json for an arbitrary package name passed in as an argument to the script. I'm guessing @nj48 used a script to go over the official list of unpublished modules and attempt to generate a placeholder for each of them. The same user also published some actual (unmanipulated) forks of the original modules, so I'm guessing this is mostly a quick move to preempt any malicious hijacking by others. There is no reason to assume the "hijacking" is malicious. Certainly not in the scripts. The user is active on GitHub and shows no indication of malicious intent. However it IS worth pointing out that the unpublished modules should now be treated with caution if you still rely on them because even if the replacements are identical and benevolent you probably need to take action.
- rhinoceraptor 11y ago
  > a script to generate the package.json for an arbitrary package name Yep, that and it publishes your project to NPM!!! edit: I was wrong, the x.sh script looks malicious but appears not to be? In any case it shows that installing NPM modules is just as unsafe as 'curl | sh'. Ways to be safe are to first check what scripts will be run by a package: 'npm show $MODULE scripts'. There is also the '--ignore-scripts' flag[1]. [1]: https://blog.liftsecurity.io/2015/01/27/a-malicious-module-on-npm https://blog.liftsecurity.io/2015/01/27/a-malicious-module-o...
  - jessaustin 11y ago
    If you're logged in, if you call it with an argument that isn't already in use as a module name, and if you specifically enter the full path to the script (you don't have cwd on your $PATH, do you?). TFA would make more sense if it contained a warning not to run random scripts that one downloads without reading them. As it is, TFA seems like an attempt to invent malicious rumors about a completely innocent person. [EDIT:] you're still wrong; in order to be equivalent to "curl | sh", x.sh or its equivalent would have to be referenced in the "scripts" object in "package.json".
  - SlashmanX 11y ago
    You got any proof of that? I don't think there's anything malicious here. The file x is simply a list of the modules that were 'given up', and x.sh is used to loop through that list and republish them as placeholders, meaning other people can't use it for malicious purposes. The author seems to be in good standing and has even republished the original code to a few modules
zamalek 11y ago
Edit: I'm definitely wrong. The person who published these was merely parking the packages. Leaving the existing (inaccurate, see response) message as one example of how badly this could have turned out. > and the content of the files is suspicious The script seems as though it might publish your entire codebase to NPM. Sorting NPM packages by creation date[1] reveals[2] a[3] few[4] potential victims. Filtering by the ISC license also seems to work. [1]: https://libraries.io/search?order=desc&platforms=NPM&sort=created_at https://libraries.io/search?order=desc&platforms=NPM&sort=cr... [2]: https://libraries.io/npm/alaska-dev https://libraries.io/npm/alaska-dev - internally hosted repo, edit: license doesn't match other projects by the same developer [3]: https://libraries.io/npm/b3app-prototype https://libraries.io/npm/b3app-prototype - private bitbucket repo, edit: deleted from npmjs [4]: https://libraries.io/npm/nodework https://libraries.io/npm/nodework - private bitbucket repo
- zwily 11y ago
  Installing one of these packages doesn't execute the script though, you'd have to do that yourself. (It could be updated to do that though, which is terrifying.)
  - zamalek 11y ago
    I wonder if the author is possibly proving a point. Regardless, it seems as though having an NPM user set up on your CI server is a big risk.
    - NathanKP 11y ago
      You have to have an NPM user setup on your CI server in order to use private NPM packages, which some companies do instead of hosting their own onsite NPM.
  - SlashmanX 11y ago
    Well there is pre/post install scripts that can be run. Not saying this package does that, but it is very easy to run a script just after installing from npm
raesene4 11y ago
A big problem with Software repositories that don't allow for /enforce cryptographic signing by the developer is that this can happen... Ideally the developer would sign before publishing and the consumer could check the signature to validate before using. Whilst not a silver bullet this is a kind of essential part of a secure package management solution.
- pvg 11y ago
  Plenty of repositories require signatures.
  - raesene4 11y ago
    for NPM? As far as I'm aware it's not even an available feature. None of rubygems/PyPi/NuGet require digital signatures... What repositories were you thinking of that do require that?
    - pvg 11y ago
      NPM doesn't. Maven does. Debian has authentication of the repository itself.
      - raesene9 11y ago
        yep the linux repositories are generally way ahead of the programming language lib ones in this regard (evidently with the exception of Maven), one of the reasons that it's a shame to see newer ones not learn the lessons that previous repo's have on security
tomku 11y ago
Looks like the author of the modules wrote a shell script to generate a package.json file and publish empty modules to npm to grab up all the unpublished names. However, when they ran it, they ran it in the same folder with their shell script and list of available modules and `npm publish` included them in the published modules as it does by default.
kylehotchkiss 11y ago
Noticed another random package was uninstalled from NPM. Please oh please don't let this thing become a trend.
jimrandomh 11y ago
There's a lesson to be drawn here about dependency hygiene. On one hand, code-reuse is a good thing, and incorporating other projects by reference is a way to make it happen. On the other hand, each dependency you add creates a little bit of risk: that the package will be updated in a way that breaks your application, or the package maintainer will go rogue, or the package will get hijacked. Most programming-language communities manage the code-reuse/dependency-hygiene tradeoff by concentrating code into a relatively small number of large libraries, and when people want a function or two that isn't in one of the libraries they're using, they incorporate it by copy-paste. In the Javascript/NPM world, on the other hand, I see a lot of projects with dependency references to huge numbers of tiny dependencies. Today, we're seeing one of the reasons why that's a liability. Most people will take away lessons about code signing and signatures, and presumably the NPM software is going to be improved in that regard. But the other lesson is that projects should have fewer dependencies. Using a library may be as simple as adding one line to a package configuration file, but using a library properly requires substantial due diligence on the library, its license, its bug tracker and its author.
- draw_down 11y ago
  No, no. This is about whether people should be able to unpublish from the registry, or how that should be handled since it's problematic in any case.
  - Xiol 11y ago
    God, I wish I could downvote you. As it is, this is the first time I've logged in for about 12 months, just to berate you.
  - ntpeters 11y ago
    Honestly, I think you're both correct. It's definitely dangerous to allow packages to be unpublished, but it can potentially be just as dangerous for people to blindly include a library without looking into a bit first. First lesson learned is of course in regard to how package managers such as NPM should handle scenarios like this. However, I would also hope this might make some people take a harder look at their dependencies to see if everything they are referencing is both truly needed and trustworthy.
askyourmother 11y ago
There are some very shaky bits in the JS dev arena. Just a little earlier, we read about how a developer unpublished his libraries from NPM, the fallout from this affected numerous high profile projects like node.js, which were relying on a left pad "library" that basically was a small string padding function.
- SlashmanX 11y ago
  This is the same story... It's a module that the developer unpublished
dontscale 11y ago
For those of us who are pissed that this is going down but need the status to get on with our lives: nj48 is a known friendly who has identified himself to us. We're going to clarify later today. https://twitter.com/seldo/status/712673227630313472 https://twitter.com/seldo/status/712673227630313472
- callmevlad 11y ago
  Yes, as far as I've heard, this was done defensively to prevent malicious actors from claiming these packages. Also, relevant conversation here in yesterday's thread: https://news.ycombinator.com/item?id=11340510 https://news.ycombinator.com/item?id=11340510
- newman314 11y ago
  I bet the orig dev was considered a known friendly until he decided to unpublish. Relying on the notion of a "known friendly" to protect packages and namespace does not strike me as a sound practice. As others may have mentioned, publishing packages really should be fire and forget. If something bad goes out, a replacement should be sent out. And for the life of me, I don't understand why they did not go with the <author/package> scheme.
kofejnik 11y ago
PSA: you should be keeping your dependencies in git
- davexunit 11y ago
  Bundling is not the answer.
  - bazqux2 11y ago
    It worked for me. After countless injections of errors after updating a large list dependencies we made the decision to stop updating and freeze the versions, patch remaining errors ourselves and leave it alone. Worked so well we now have time to rebuild the whole front end on a new stack as an experiment instead wasting time with endless bug hunts.
    - davexunit 11y ago
      Maybe bundling is fine for your in-house proprietary software, but it's absolutely not OK for free software where users and administrators need to keep on top of things like security updates. When projects bundle their dependencies, users become dependent on that project to provide critical updates to software that the project didn't even write. This multiplies for each piece of software that bundles their dependencies. It's simply unsustainable and irresponsible.
      - bazqux2 11y ago
        I agree. Someone making free OS software for others to use shouldn't bundle. I made the assumption on the top post that they were in-house proprietary software given the reference to keeping everything in git.
        davexunit 11y ago
        I guess we're on the same page! Sorry!
_Marak_ 11y ago
I've been on the receiving end of a NPM name dispute violation handed down by Izs. https://docs.npmjs.com/misc/disputes https://docs.npmjs.com/misc/disputes Was on vacation and found out that I had lost a published package name within 24 hours of the dispute request. Broke a few production systems. Really messed up my day. Ended up having to beg with the person who filed the original request and they eventually gave me the package back. Honestly, the whole process was a bit personal and I felt like I was being singled out as an individual by NPM, rather than being treated like a developer who was using the service. Not a nice feeling.
- anon4 11y ago
  Also was there even an actual trademark violation here? Sounds like the entire thing shouldn't have happened in the first place. Of course, people will take home the lesson "Koçulu is a crazy guy who cares more about stupid ideology than keeping things working" rather than "Kik lawyers are way too trigger-happy" and "Holy shit I can just unpublish and replace whichever module I want by just filing an unrelated trademark and emailing one guy".
drinchev 11y ago
Author of the post here. According to a tweet [1], the user @nj48 seems to be non-malicious. I updated the blog post. Nevertheless I find his actions dangerous and irresponsible. [1] https://twitter.com/seldo/status/712673227630313472 https://twitter.com/seldo/status/712673227630313472
- callmevlad 11y ago
  Given the conversation in yesterday's thread [1], I think a known (and personally identified) non-malicious person defensively reserving these packages is less dangerous than some unknown entity taking over these packages later. I personally think the best course of action would have been for the NPM team to immediately blacklist these names (aside from left-pad, that's a separate conversation) after the entire list of unpublished packages was shared, and then make them available on a case-by-case basis. https://news.ycombinator.com/item?id=11340510 https://news.ycombinator.com/item?id=11340510
- tombrossman 11y ago
  Curious, should those of us using their Linux PPAs disable them until this is sorted? The idea of re-using disabled package names is out there now and I wonder how long it is before someone malicious tries this. Is this an unreasonable concern?
  - drinchev 11y ago
    Not sure about that. I'm away of the PPA's + NodeJS's NPM packaging model.
dustinmoorenet 11y ago
As shitty as unpublishing is, I think the best thing the npm community needs to take away from this is auto updating is bad. Remove the ~ and ^ from your dependencies so that only a package of a specific version can be installed. "This or better" thinking doesn't work if "better" is unknown. I know that the version I am using currently is fine but I don't know about future updates. Even if we had signed packages, we are still installing unknown software if we just trust other devs. I lock my dependency version and then use https://www.npmjs.com/package/npm-check-updates https://www.npmjs.com/package/npm-check-updates to figure out what needs updating, then I test my code. This should not be done as an automatic part of the build process. *edited npm capitalization
- Klathmon 11y ago
  No need to muck about with your package.JSON, actually fix the problem by just checking in your node_modules folder. Then continue working like you always have. There are some changes that you'll need of you are using native modules, but they are simple and easy to do.
  - AgentME 11y ago
    You can use shrinkpack[1] to solve the issues with native modules. It allows you to check in the archives that npm downloads instead of the installed node_modules folder. [1]: https://github.com/JamieMason/shrinkpack https://github.com/JamieMason/shrinkpack
    - Klathmon 11y ago
      That's a great solution, but i was just talking about installing the source into node_modules and running `npm rebuild` instead of `npm install` when you install your app.
- kylehotchkiss 11y ago
  By adding `save-exact=true` to .npmrc in a project NPM saves the exact version. Learned this trying to fix Shrinkwrap a few weeks back but seems like it's a good practice for securities sake now.
BinaryBullet 11y ago
In the root project folder (containing package.json and ./node_modules), you can run the following: comm -12 <(ls ./node_modules) <(curl https://gist.githubusercontent.com/azer/db27417ee84b5f34a6ea/raw/50ab7ef26dbde2d4ea52318a3590af78b2a21162/gistfile1.txt) It will output any of the modules that @azer unpublished yesterday (that are being used by your project).
- BinaryBullet 11y ago
  NOTE: this only works for NPM 3's flat node_module structure
ErikAugust 11y ago
Anything wrong with checking your dependencies into your repo? That way, you know, they can't go away or be hijacked.
gmisra 11y ago
It's worth noting that the official NPM dispute resolution policy makes the following very clear (https://docs.npmjs.com/misc/disputes https://docs.npmjs.com/misc/disputes) > Some things are not allowed, and will be removed without discussion if they are brought to the attention of the npm registry admins, including but not limited to: ... 4. "Squatting" on a package name that you plan to use, but aren't actually using. Sorry, I don't care how great the name is, or how perfect a fit it is for the thing that someday might happen. If someone wants to use it today, and you're just taking up space with an empty tarball, you're going to be evicted. 5. Putting empty packages in the registry. Packages must have SOME functionality. It can be silly, but it can't be nothing. (See also: squatting.)
heavenlyhash 11y ago
I challenge someone to tell me the difference between npm and an RCE. "npm" is two things in this picture: the server and centralized service, and the software tool on everyone's build/dev machines. Folks have historically been happy trusting the centralized npm server to behave consistently and pleasantly, and some opinions have recently shifted on that. But frankly, the server doesn't matter, in the big picture. The npm tool on your computer does. It's the one executing code on your computer with all of your local user's privileges. This tool starts executing new code from a new author on my host as my user without any authentication except "trust the server". This is exactly the same words we would use to describe behavior of $script_kiddie_virus_of_the_week: > "download code from C&C server; run it, thanks:D" What's the difference here? "good intentions"? I'd rather have something more than "good intentions" controlling what code ends up running on my computer. Wouldn't you?
ktRolster 11y ago
As a rule of thumb, if a dependency would take you less than two days to rewrite (one day for writing, one day for testing), it's better to do it yourself and avoid the problems of dependencies.
- spriggan3 11y ago
  How many nodejs developers actually audit their dependencies ? In my experience not many. it's just "npm search" without thinking. then "my problem is somebody else now". Frankly the whole ecosystem is just scary.
odinduty 11y ago
Name squatting in those package managers is a real problem. I found this on pypi and it was a very unpleasant surprise. https://sourceforge.net/p/pypi/support-requests/571/ https://sourceforge.net/p/pypi/support-requests/571/
spriggan3 11y ago
Sorry but the elephant in the room IS the lack of namespacing. There is no namespacing on NPM. If there were something like that this issue would have never happened. This is not a problem of distributed vs non distributed, NPM doesn't need a "blockchain" or whatever. Packages SHOULD by namespaced period and any serious package manager uses namespaces. I (and many others) called it from the very beginning and warned NPM authors that the lack of namespace would eventually lead to this kind of issue. NPM authors didn't give a damn ( there are other issues that have been known for years but they fell on deaf ears ). Packages should be resolved by a namespace + the name of the package. Now everybody is in panic mode because nobody knows what one is fetching from NPM anymore. Now I hope NPM author will come to their senses and change the way NPM works. But the trust is broken, no question. Between stuff like that, people selling "realestate" on NPM for real money... Nodejs has been the least professional platform I have ever used. Everybody's out there to make a quick buck, nobody gives a damn, this is whole thing will collapse sooner or latter. Finally people need to stop with the "unix philosophy" excuse. Importing 10 lines of code from a random package is not the "unix philosophy". Splitting a 100 loc module into 20 packages is not the "unix philosophy". Packages have so many dependencies it's getting ridiculous. edit: corrected
- joslin01 11y ago
  Sorry, 100% agree with you but with your life going forward, it's "fell on deaf ears"
  - spriggan3 11y ago
    Oh, sorry, I'm not a native English speaker so yeah I tend to make stupid mistakes like these.

238 points by vdfs 11y ago

It's insane that NPM allows anybody to just take over a module name if the original is unpublished, without even a warning to users.

henrikfr 11y ago
Insane, yes, but I guess it's just never really been that much of a problem until now. Hopefully something good will come out of this..
- smt88 11y ago
  > it's just never really been that much of a problem until now That's a reasonable excuse in the 90s, before automatic updates over HTTP were common. Our industry now has decades of experience securing HTTP updates and package managers, with various Linux solutions demonstrating good practices.
- CorpOverreach 11y ago
  You would think this is part of security 101 for these things... And people wonder why big enterprises are scared of touching open source stuff.
  - nyan4 11y ago
    > And people wonder why big enterprises are scared of touching open source stuff. some open source stuff. Most enterprises dig distributions, especially with LTS.
  - cyphar 11y ago
    Many big enterprises run on free software. Just because this particular free software project is a shit-show doesn't make all free software projects this bad.
- gst 11y ago
  Something similar is (or at least was) true for Github: I changed my username a couple of years ago and I recently found out that another user is now using my original username. Less of an issue due to the much longer timeframe, but I think not allowing username re-use would be a safer choice.
- FranOntanaya 11y ago
  Module repositories have been a thing for decades. Even a platform as crusty-old as WordPress doesn't allow repository take-overs for plugins. Surely someone had some experience outside npm to think about this.
na85 11y ago
Insane, yes, but not unexpected. This is the sort of amateur behaviour that is emblematic of the entire nodejs community.
- akerro 11y ago
  That's what happens when a language and framework have very low entry level, more and more not responsible people gather in one place. Yes, they make it popular and trendy, but they're dangerous for themselves and others.
  - petetnt 11y ago
    Amazingly shameful cheap shots and mudslinging against hundreds of people from both you and @na85. Bravo.
    - stonogo 11y ago
      Those people deserve criticism, because they're making bad decisions.
      - sangnoir 11y ago
        and by 'Those people', you mean the entire NodeJS community? > This is the sort of amateur behaviour that is emblematic of the entire nodejs community. This is pure mudslinging
        akerro 11y ago
        No, I'm sure he didn't meant whole community. There was a reason why njs became so popular, and there are great people behind it, but as everything that's easy to start with, it attracts people with no skills, no experience. When learning C one of the first things you learn is how operating systems work, what memory leaks are, why they are dangerous, C, C++ have higher entry level, people who stay longer with C/C++ have very good experience, they know more about security, risks, standards, systems... When you have low entry lever, you don't even bother reading on wiki what XSS is, what .ssh folder is. We just learnt the hard way that package manager for NJS is crap, just to compare it, take a look at crates of a language that has high entry level: https://www.reddit.com/r/rust/comments/4bm3rk/how_would_cratesio_react_in_a_case_similar_to_the/ https://www.reddit.com/r/rust/comments/4bm3rk/how_would_crat...
viperscape 11y ago
https://twitter.com/_scape/status/700453739182817281 https://twitter.com/_scape/status/700453739182817281 My semi relevant tweet, it's just asking for it with that warning
- irl_zebra 11y ago
  Why not just post the tweet here instead of spamming your Twitter feed?
  - viperscape 11y ago
    That's a perma link to the direct tweet... why paste in the tweet when I can just link it...
    - prodigal_erik 11y ago
      HN would be unusable if everyone stopped writing comments and just pasted links to an ephemeral third-party service.
    - aw3c2 11y ago
      http://pastebin.com/XkPwwkUs http://pastebin.com/XkPwwkUs
    - douche 11y ago
      Bravo, you've illustrated the whole problem
drdaeman 11y ago
Link rot and domain squatting are a well-known decades-old issues. NPM identifiers can be considered as just another weird form of pseudo-URIs and aren't special here. It is insane that modules don't have signatures (that are actually verified, of course). Because npm can feed you basically anything. It's not a problem that something else gets published under the old address. It's perfectly natural. The real problem is the trust model - that new content's accepted without even warnings.
choward 11y ago
It's insane that NPM allows anybody to un-publish modules to begin with. Even if I have a dependency locked down in my NPM shrinkwrap file, it can change underneath me? That is pretty absurd and gives me zero confidence in my packages. It means I MUST commit them to source control or risk having my project completely broken some day. I thought for sure that since NPM removed the ability to re-publish the same version of a package with different content that they also wouldn't let you remove versions of a package. It also means you should never user the "^" version specifier or risk downloading some completely different project. They do have a "warning" at https://docs.npmjs.com/cli/unpublish https://docs.npmjs.com/cli/unpublish: > WARNING > > It is generally considered bad behavior to remove versions of a library that others are depending on! Seriously, what good does that do? Nobody takes warnings seriously.
- Wintamute 11y ago
  A specific version of a package is immutable on npm. So if you depend on exactly foo@1.2.3 in your shrinkwrap you'll always get the same code. In other words, even if the package is unpublished, and picked up by a new user they can't re-publish new code to 1.2.3. If you depend on a version range like ^1.2.3 or ~1.2.3 its a different story, of course. Moral of the story is imo always pin to exact versions and use shrinkwrap for production apps. EDIT: This may not be true .. see below.
  - choward 11y ago
    > A specific version of a package is immutable on NPM. That's not true. I thought it was but it's not. That's the terrible part. To demonstrate with one of the packages that was removed, run: $ npm info andthen You'll see that versions 0.0.1 and 0.0.2 were published at one point. However, for "versions" it only mentions 2.0.0. And of course, if you run: $ npm install andthen@0.0.2 It blows up in your face.
    - Wintamute 11y ago
      :o

pluma 11y ago

Overly dramatic. The content is not suspicious at all, it's clearly a script to generate the package.json for an arbitrary package name passed in as an argument to the script. I'm guessing @nj48 used a script to go over the official list of unpublished modules and attempt to generate a placeholder for each of them. The same user also published some actual (unmanipulated) forks of the original modules, so I'm guessing this is mostly a quick move to preempt any malicious hijacking by others. There is no reason to assume the "hijacking" is malicious. Certainly not in the scripts. The user is active on GitHub and shows no indication of malicious intent. However it IS worth pointing out that the unpublished modules should now be treated with caution if you still rely on them because even if the replacements are identical and benevolent you probably need to take action.

rhinoceraptor 11y ago
> a script to generate the package.json for an arbitrary package name Yep, that and it publishes your project to NPM!!! edit: I was wrong, the x.sh script looks malicious but appears not to be? In any case it shows that installing NPM modules is just as unsafe as 'curl | sh'. Ways to be safe are to first check what scripts will be run by a package: 'npm show $MODULE scripts'. There is also the '--ignore-scripts' flag[1]. [1]: https://blog.liftsecurity.io/2015/01/27/a-malicious-module-on-npm https://blog.liftsecurity.io/2015/01/27/a-malicious-module-o...
- jessaustin 11y ago
  If you're logged in, if you call it with an argument that isn't already in use as a module name, and if you specifically enter the full path to the script (you don't have cwd on your $PATH, do you?). TFA would make more sense if it contained a warning not to run random scripts that one downloads without reading them. As it is, TFA seems like an attempt to invent malicious rumors about a completely innocent person. [EDIT:] you're still wrong; in order to be equivalent to "curl | sh", x.sh or its equivalent would have to be referenced in the "scripts" object in "package.json".
- SlashmanX 11y ago
  You got any proof of that? I don't think there's anything malicious here. The file x is simply a list of the modules that were 'given up', and x.sh is used to loop through that list and republish them as placeholders, meaning other people can't use it for malicious purposes. The author seems to be in good standing and has even republished the original code to a few modules

zamalek 11y ago

Edit: I'm definitely wrong. The person who published these was merely parking the packages. Leaving the existing (inaccurate, see response) message as one example of how badly this could have turned out. > and the content of the files is suspicious The script seems as though it might publish your entire codebase to NPM. Sorting NPM packages by creation date[1] reveals[2] a[3] few[4] potential victims. Filtering by the ISC license also seems to work. [1]: https://libraries.io/search?order=desc&platforms=NPM&sort=created_at https://libraries.io/search?order=desc&platforms=NPM&sort=cr... [2]: https://libraries.io/npm/alaska-dev https://libraries.io/npm/alaska-dev - internally hosted repo, edit: license doesn't match other projects by the same developer [3]: https://libraries.io/npm/b3app-prototype https://libraries.io/npm/b3app-prototype - private bitbucket repo, edit: deleted from npmjs [4]: https://libraries.io/npm/nodework https://libraries.io/npm/nodework - private bitbucket repo

zwily 11y ago
Installing one of these packages doesn't execute the script though, you'd have to do that yourself. (It could be updated to do that though, which is terrifying.)
- zamalek 11y ago
  I wonder if the author is possibly proving a point. Regardless, it seems as though having an NPM user set up on your CI server is a big risk.
  - NathanKP 11y ago
    You have to have an NPM user setup on your CI server in order to use private NPM packages, which some companies do instead of hosting their own onsite NPM.
- SlashmanX 11y ago
  Well there is pre/post install scripts that can be run. Not saying this package does that, but it is very easy to run a script just after installing from npm

raesene4 11y ago

A big problem with Software repositories that don't allow for /enforce cryptographic signing by the developer is that this can happen... Ideally the developer would sign before publishing and the consumer could check the signature to validate before using. Whilst not a silver bullet this is a kind of essential part of a secure package management solution.

pvg 11y ago
Plenty of repositories require signatures.
- raesene4 11y ago
  for NPM? As far as I'm aware it's not even an available feature. None of rubygems/PyPi/NuGet require digital signatures... What repositories were you thinking of that do require that?
  - pvg 11y ago
    NPM doesn't. Maven does. Debian has authentication of the repository itself.
    - raesene9 11y ago
      yep the linux repositories are generally way ahead of the programming language lib ones in this regard (evidently with the exception of Maven), one of the reasons that it's a shame to see newer ones not learn the lessons that previous repo's have on security

tomku 11y ago

Looks like the author of the modules wrote a shell script to generate a package.json file and publish empty modules to npm to grab up all the unpublished names. However, when they ran it, they ran it in the same folder with their shell script and list of available modules and `npm publish` included them in the published modules as it does by default.

kylehotchkiss 11y ago

Noticed another random package was uninstalled from NPM. Please oh please don't let this thing become a trend.

jimrandomh 11y ago

There's a lesson to be drawn here about dependency hygiene. On one hand, code-reuse is a good thing, and incorporating other projects by reference is a way to make it happen. On the other hand, each dependency you add creates a little bit of risk: that the package will be updated in a way that breaks your application, or the package maintainer will go rogue, or the package will get hijacked. Most programming-language communities manage the code-reuse/dependency-hygiene tradeoff by concentrating code into a relatively small number of large libraries, and when people want a function or two that isn't in one of the libraries they're using, they incorporate it by copy-paste. In the Javascript/NPM world, on the other hand, I see a lot of projects with dependency references to huge numbers of tiny dependencies. Today, we're seeing one of the reasons why that's a liability. Most people will take away lessons about code signing and signatures, and presumably the NPM software is going to be improved in that regard. But the other lesson is that projects should have fewer dependencies. Using a library may be as simple as adding one line to a package configuration file, but using a library properly requires substantial due diligence on the library, its license, its bug tracker and its author.

draw_down 11y ago
No, no. This is about whether people should be able to unpublish from the registry, or how that should be handled since it's problematic in any case.
- Xiol 11y ago
  God, I wish I could downvote you. As it is, this is the first time I've logged in for about 12 months, just to berate you.
- ntpeters 11y ago
  Honestly, I think you're both correct. It's definitely dangerous to allow packages to be unpublished, but it can potentially be just as dangerous for people to blindly include a library without looking into a bit first. First lesson learned is of course in regard to how package managers such as NPM should handle scenarios like this. However, I would also hope this might make some people take a harder look at their dependencies to see if everything they are referencing is both truly needed and trustworthy.

askyourmother 11y ago

There are some very shaky bits in the JS dev arena. Just a little earlier, we read about how a developer unpublished his libraries from NPM, the fallout from this affected numerous high profile projects like node.js, which were relying on a left pad "library" that basically was a small string padding function.

SlashmanX 11y ago
This is the same story... It's a module that the developer unpublished

dontscale 11y ago

For those of us who are pissed that this is going down but need the status to get on with our lives: nj48 is a known friendly who has identified himself to us. We're going to clarify later today. https://twitter.com/seldo/status/712673227630313472 https://twitter.com/seldo/status/712673227630313472

callmevlad 11y ago
Yes, as far as I've heard, this was done defensively to prevent malicious actors from claiming these packages. Also, relevant conversation here in yesterday's thread: https://news.ycombinator.com/item?id=11340510 https://news.ycombinator.com/item?id=11340510
newman314 11y ago
I bet the orig dev was considered a known friendly until he decided to unpublish. Relying on the notion of a "known friendly" to protect packages and namespace does not strike me as a sound practice. As others may have mentioned, publishing packages really should be fire and forget. If something bad goes out, a replacement should be sent out. And for the life of me, I don't understand why they did not go with the <author/package> scheme.

kofejnik 11y ago

PSA: you should be keeping your dependencies in git

davexunit 11y ago
Bundling is not the answer.
- bazqux2 11y ago
  It worked for me. After countless injections of errors after updating a large list dependencies we made the decision to stop updating and freeze the versions, patch remaining errors ourselves and leave it alone. Worked so well we now have time to rebuild the whole front end on a new stack as an experiment instead wasting time with endless bug hunts.
  - davexunit 11y ago
    Maybe bundling is fine for your in-house proprietary software, but it's absolutely not OK for free software where users and administrators need to keep on top of things like security updates. When projects bundle their dependencies, users become dependent on that project to provide critical updates to software that the project didn't even write. This multiplies for each piece of software that bundles their dependencies. It's simply unsustainable and irresponsible.
    - bazqux2 11y ago
      I agree. Someone making free OS software for others to use shouldn't bundle. I made the assumption on the top post that they were in-house proprietary software given the reference to keeping everything in git.
      - davexunit 11y ago
        I guess we're on the same page! Sorry!

_Marak_ 11y ago

I've been on the receiving end of a NPM name dispute violation handed down by Izs. https://docs.npmjs.com/misc/disputes https://docs.npmjs.com/misc/disputes Was on vacation and found out that I had lost a published package name within 24 hours of the dispute request. Broke a few production systems. Really messed up my day. Ended up having to beg with the person who filed the original request and they eventually gave me the package back. Honestly, the whole process was a bit personal and I felt like I was being singled out as an individual by NPM, rather than being treated like a developer who was using the service. Not a nice feeling.

anon4 11y ago
Also was there even an actual trademark violation here? Sounds like the entire thing shouldn't have happened in the first place. Of course, people will take home the lesson "Koçulu is a crazy guy who cares more about stupid ideology than keeping things working" rather than "Kik lawyers are way too trigger-happy" and "Holy shit I can just unpublish and replace whichever module I want by just filing an unrelated trademark and emailing one guy".

drinchev 11y ago

Author of the post here. According to a tweet [1], the user @nj48 seems to be non-malicious. I updated the blog post. Nevertheless I find his actions dangerous and irresponsible. [1] https://twitter.com/seldo/status/712673227630313472 https://twitter.com/seldo/status/712673227630313472

callmevlad 11y ago
Given the conversation in yesterday's thread [1], I think a known (and personally identified) non-malicious person defensively reserving these packages is less dangerous than some unknown entity taking over these packages later. I personally think the best course of action would have been for the NPM team to immediately blacklist these names (aside from left-pad, that's a separate conversation) after the entire list of unpublished packages was shared, and then make them available on a case-by-case basis. https://news.ycombinator.com/item?id=11340510 https://news.ycombinator.com/item?id=11340510
tombrossman 11y ago
Curious, should those of us using their Linux PPAs disable them until this is sorted? The idea of re-using disabled package names is out there now and I wonder how long it is before someone malicious tries this. Is this an unreasonable concern?
- drinchev 11y ago
  Not sure about that. I'm away of the PPA's + NodeJS's NPM packaging model.

dustinmoorenet 11y ago

As shitty as unpublishing is, I think the best thing the npm community needs to take away from this is auto updating is bad. Remove the ~ and ^ from your dependencies so that only a package of a specific version can be installed. "This or better" thinking doesn't work if "better" is unknown. I know that the version I am using currently is fine but I don't know about future updates. Even if we had signed packages, we are still installing unknown software if we just trust other devs. I lock my dependency version and then use https://www.npmjs.com/package/npm-check-updates https://www.npmjs.com/package/npm-check-updates to figure out what needs updating, then I test my code. This should not be done as an automatic part of the build process. *edited npm capitalization

Klathmon 11y ago
No need to muck about with your package.JSON, actually fix the problem by just checking in your node_modules folder. Then continue working like you always have. There are some changes that you'll need of you are using native modules, but they are simple and easy to do.
- AgentME 11y ago
  You can use shrinkpack[1] to solve the issues with native modules. It allows you to check in the archives that npm downloads instead of the installed node_modules folder. [1]: https://github.com/JamieMason/shrinkpack https://github.com/JamieMason/shrinkpack
  - Klathmon 11y ago
    That's a great solution, but i was just talking about installing the source into node_modules and running `npm rebuild` instead of `npm install` when you install your app.
kylehotchkiss 11y ago
By adding `save-exact=true` to .npmrc in a project NPM saves the exact version. Learned this trying to fix Shrinkwrap a few weeks back but seems like it's a good practice for securities sake now.

BinaryBullet 11y ago

In the root project folder (containing package.json and ./node_modules), you can run the following: comm -12 <(ls ./node_modules) <(curl https://gist.githubusercontent.com/azer/db27417ee84b5f34a6ea/raw/50ab7ef26dbde2d4ea52318a3590af78b2a21162/gistfile1.txt) It will output any of the modules that @azer unpublished yesterday (that are being used by your project).

BinaryBullet 11y ago
NOTE: this only works for NPM 3's flat node_module structure

ErikAugust 11y ago

Anything wrong with checking your dependencies into your repo? That way, you know, they can't go away or be hijacked.

gmisra 11y ago

It's worth noting that the official NPM dispute resolution policy makes the following very clear (https://docs.npmjs.com/misc/disputes https://docs.npmjs.com/misc/disputes) > Some things are not allowed, and will be removed without discussion if they are brought to the attention of the npm registry admins, including but not limited to: ... 4. "Squatting" on a package name that you plan to use, but aren't actually using. Sorry, I don't care how great the name is, or how perfect a fit it is for the thing that someday might happen. If someone wants to use it today, and you're just taking up space with an empty tarball, you're going to be evicted. 5. Putting empty packages in the registry. Packages must have SOME functionality. It can be silly, but it can't be nothing. (See also: squatting.)

heavenlyhash 11y ago

I challenge someone to tell me the difference between npm and an RCE. "npm" is two things in this picture: the server and centralized service, and the software tool on everyone's build/dev machines. Folks have historically been happy trusting the centralized npm server to behave consistently and pleasantly, and some opinions have recently shifted on that. But frankly, the server doesn't matter, in the big picture. The npm tool on your computer does. It's the one executing code on your computer with all of your local user's privileges. This tool starts executing new code from a new author on my host as my user without any authentication except "trust the server". This is exactly the same words we would use to describe behavior of $script_kiddie_virus_of_the_week: > "download code from C&C server; run it, thanks:D" What's the difference here? "good intentions"? I'd rather have something more than "good intentions" controlling what code ends up running on my computer. Wouldn't you?

ktRolster 11y ago

As a rule of thumb, if a dependency would take you less than two days to rewrite (one day for writing, one day for testing), it's better to do it yourself and avoid the problems of dependencies.

spriggan3 11y ago
How many nodejs developers actually audit their dependencies ? In my experience not many. it's just "npm search" without thinking. then "my problem is somebody else now". Frankly the whole ecosystem is just scary.

odinduty 11y ago

Name squatting in those package managers is a real problem. I found this on pypi and it was a very unpleasant surprise. https://sourceforge.net/p/pypi/support-requests/571/ https://sourceforge.net/p/pypi/support-requests/571/

spriggan3 11y ago

Sorry but the elephant in the room IS the lack of namespacing. There is no namespacing on NPM. If there were something like that this issue would have never happened. This is not a problem of distributed vs non distributed, NPM doesn't need a "blockchain" or whatever. Packages SHOULD by namespaced period and any serious package manager uses namespaces. I (and many others) called it from the very beginning and warned NPM authors that the lack of namespace would eventually lead to this kind of issue. NPM authors didn't give a damn ( there are other issues that have been known for years but they fell on deaf ears ). Packages should be resolved by a namespace + the name of the package. Now everybody is in panic mode because nobody knows what one is fetching from NPM anymore. Now I hope NPM author will come to their senses and change the way NPM works. But the trust is broken, no question. Between stuff like that, people selling "realestate" on NPM for real money... Nodejs has been the least professional platform I have ever used. Everybody's out there to make a quick buck, nobody gives a damn, this is whole thing will collapse sooner or latter. Finally people need to stop with the "unix philosophy" excuse. Importing 10 lines of code from a random package is not the "unix philosophy". Splitting a 100 loc module into 20 packages is not the "unix philosophy". Packages have so many dependencies it's getting ridiculous. edit: corrected

joslin01 11y ago
Sorry, 100% agree with you but with your life going forward, it's "fell on deaf ears"
- spriggan3 11y ago
  Oh, sorry, I'm not a native English speaker so yeah I tend to make stupid mistakes like these.