22 ms·

Malicious code in the purescript NPM installer

521 points by braythwayt 7y ago

svnpenn 7y ago
The real issue the Balkanization of JavaScript programs. The `rate-map` package is essentially one line of code: start + val * (end - start); https://github.com/shinnn/rate-map/blob/90c234c9/index.mjs#L38 https://github.com/shinnn/rate-map/blob/90c234c9/index.mjs#L...
- SCLeo 7y ago
  I honestly don't understand why people use packages like this. If I need this functionality, I will simply write my own. Plus, I will never able to find this specific package. I guess PureScript uses this because its author is also the author of rate-map.
  - ricardobeat 7y ago
    I’ll give you one: it’s code already written and tested by > 1 person, edge cases already figured out. Saves you time. The gains are small but quickly add up. This is why lately I’ve been a fan of very extensive standard libraries (like Crystal has) - its like having a huge repository but vetoed by the same team and without any of the package management drawbacks.
    - spenczar5 7y ago
      > The gains are small but quickly add up. There are costs to these micro-libraries that outweigh the gains. This code is trivial; there aren’t really edge cases to be worked out.
      - jasonhansel 7y ago
        Also: the way the library handles those edge cases isn't necessarily the way you want. Case in point: rate-map throws exceptions in situations where you might expect it to fail more gracefully.
    - cannedslime 7y ago
      What gains? That line wouldn't take much resources or time to figure out and test. Now you have yet another dependency that could be injected with bad code in the future...
    - vhakulinen 7y ago
      > it’s code already written and tested by > 1 person, edge cases already figured out. It is tested? Edge cases figured out? Essentially for this code?: return start + val * (end - start); Sure, if its running in hostile environment, you might need to do those sanity checks for your parameters - but I have hard time imagining such situation. If you actually need to "Map a number in the range of 0-1 to a new value with a given range" in your own code, can't you guarantee that the variables are all numbers? Its your responsibility as a developer to know your code, and the data your code is handling. > Saves you time. There is something really wrong if finding a package to do this niche thing is faster and more optimal that just writing out that one line of code.
    - inimino 7y ago
      [C]ode already written and tested by > 1 person, edge cases already figured out wastes time when that person was not you. This costs you time. The losses are small but they add up.
    - tjpnz 7y ago
      >The gains are small but quickly add up. Same could also be said for build times not to mention security issues should one of your micro-dependencies be hijacked.
    - ben509 7y ago
      > edge cases already figured out. Nope, it doesn't guarantee its invariants. This would pass all the tests and yet returns a value out of bounds: > s = 1e12 1000000000000 > e = 1e-8 1e-8 > s + 1.0 * (e - s) 0
    - dangoor 7y ago
      > This is why lately I’ve been a fan of very extensive standard libraries (like Crystal has) - its like having a huge repository but vetoed by the same team and without any of the package management drawbacks. Having had some involvement in the early days of node, I had imagined there being something like the Python stdlib. When the npm world grew, I thought "oh, that's pretty cool. It's neat how npm can handle multiple versions of the same package." Now, I'm in absolute agreement with you. There are definitely downsides to a large standard library, but I think the upsides are worth it if that library is maintained.
  - dmitriid 7y ago
    Funnily enough, in my opinion, the JS ecosystem fully embraced the Unix way: have small programs/libs that do only one thing and do it (somewhat) well.
    - lenkite 7y ago
      I fear your analogy does not go far enough if you are comparing the JS ecosystem to the UNIX philosophy. Javascript would offer a library for every single option, variant and logical operator for a UNIX command. Combinatorial explosion will devour the web. It already destroys dev laptops anyways when downloading something via NPM.
    - inimino 7y ago
      No, they do not on the whole do anything well.
  - GrumpyNl 7y ago
    You should take a look at the trim package and how often hat is downloaded.
    - reeeeee 7y ago
      There is not even a link to the source code on the npm page for it. I installed it and inspected the source code, but I doubt everyone does this when installing a dependency.
      - delfaras 7y ago
        Here's the complete source for anyone curious ``` exports = module.exports = trim; function trim(str){ return str.replace(/^\s|\s$/g, ''); } exports.left = function(str){ return str.replace(/^\s/, ''); }; exports.right = function(str){ return str.replace(/\s$/, ''); }; ```
        rjmunro 7y ago
        That code doesn't match what actually happens. It will only trim a single character. Have asterisks been trimmed in your copy-paste or something?
        delfaras 7y ago
        Ah you're right, the asterisks have been interpreted as italics. Can't edit my comment, sorry Here's a paste: https://pastebin.com/kBHprdyj https://pastebin.com/kBHprdyj
  - sergiomattei 7y ago
    As usual, everyone here will start their "I'm better than this" comments. If you've ever used a code dependency, you are a target for malicious code. That's just how it is. In this case, using small packages like this helps in... 1. Reliability - these packages are typically 100% unit tested 2. Convenience 3. Reduce codebase size. I can't imagine having to copy paste every little small function into a mega utils file.
- jasonhansel 7y ago
  ...wow. I literally did not believe that until I clicked the link. JavaScript has gone too far.
  - iamleppert 7y ago
    The problem is clearly due to vanity metrics like number of packages motivating people to publish an insane number of useless packages to fluff their contributions.
    - inimino 7y ago
      That is not clear.
    - ivanfon 7y ago
      Agreed - I’ve found and been astonished by Github users who maintain hundreds of these little npm packages, all of which have usually under 10 lines of actual code, and sometimes even have chains of dependencies on the user’s other packages. It seems like the only reason these packages get any significant downloads is when one of them gets depended on by a big package, causing the entire dependency chain of the user’s little packages to be downloaded.
  - fock 7y ago
    well, the rate-map at least includes a function: https://github.com/sindresorhus/shebang-regex https://github.com/sindresorhus/shebang-regex
    - hu3 7y ago
      A dependency for /^#!(.*)/; With 8.6 mil downloads and 71 dependents... https://www.npmjs.com/package/shebang-regex https://www.npmjs.com/package/shebang-regex
      - fock 7y ago
        and if you look closely, 70 of those dependents are just small-scale/bs packages and the only sensible real dependency is cross-spawn _via_ the shebang package (which applies this regex to a string). The whole ecosysten could use a purge of all those useless 1-line-requires (which were all introduced by helpful commits from the "i has 1337 downloads"-community), currently this is madness.
- ww520 7y ago
  But it's great for resume and portfolio.
  - s_Hogg 7y ago
    This is the problem - how to disincentivise that behaviour?
    - throwaway66666 7y ago
      Do not hire people who do this. Most of them will say "over 200m downloads of npm packages". Go and take a look. If you see things like 'is-not-foo, checks if a given string is not equal to the string "foo"', and 0 meaningful contributions, either pay no attention to these claims or pass.
- tonetheman 7y ago
  The real issue is most JS programmers are not smart enough to write it so they have to find packages to do anything.
  - dang 7y ago
    Hey, can you please stop breaking the site guidelines? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html I don't want to ban you, but when accounts keep doing this, we kind of have to.
- na85 7y ago
  It's an ecosystem full of reinventing the wheel. The most popular library, lodash, includes a reimplementation of a foreach loop for Pete's sake, for reasons passing understanding since it's part of the ecma spec. JavaScript is just amateur hour, and these things are going to keep happening. It's pathological.
  - rwbcxrz 7y ago
    Which foreach are you talking about? Array.prototype.forEach, for...in loops, for...of loops? The first only works with arrays and array-like objects. The second works on objects and arrays, but it iterates over all enumerable properties, so you don't want really want to use it for arrays. It's also made a lot less useful because it only iterates over properties, not keys. The third finally provides some sanity, but it's only been around since ES6. Before that, lodash's each method was the most reliable way to iterate over a collection, be it an object or an array. Just because you don't know the reason for something doesn't mean there isn't one.
    - runarberg 7y ago
      `for ... of` only iterate over objects that implement `Symbol.iterator`. Native objects don’t do that by default, so `_.forEach` is more useful the `for ... of` even if you are only targeting modern browsers and not compiling the code down to an earlier version of the spec. That said, you can use `Object.keys`, `Object.values`, or `Object.entries` if you want to iterate over objects that don’t implement `Symbol.iterator`, so if you only need `_.forEach` there is no reason to pull in any libraries. Object.entries(object).forEach(([key, value]) => { // ... });
    - inimino 7y ago
      > Array.prototype.forEach Yes, this one. If the object is an Array. According to whatever test they are using for that. Lodash includes a reimplementation of Array.prototype.forEach because mistakes were made. It also works on other objects because other mistakes were made. We all make mistakes. But just because there is a reason for something does not mean there is a good reason.
- aur09 7y ago
  Wow—and spread into 11 files to facilitate CI, linting, etc—and with 1kb worth of error checking. That is nuts!
- TeMPOraL 7y ago
  And this one-liner library even has a dependency (!) used by the surrounding error-checking code.
  - ddalex 7y ago
    What's the difference between an NPM dependency chain and a black hole? We can measure the depth of a black hole.
- lenkite 7y ago
  Don't know whether to laugh or cry at this. Truth is stranger than fiction in the land of JavaScript. Could any developer 20 years ago predict this is what software engineering on the Web would devolve to ?
bsamuels 7y ago
I wonder how bad will this cred-stuffing package authors problem will get before npm/other package managers flat out require 2FA for maintainers
- hn_throwaway_99 7y ago
  I think the blog author is implying as much as he can, without directly accusing, that he believes that https://github.com/shinnn https://github.com/shinnn was responsible for the bad code, not a random hack.
  - JulianMorrison 7y ago
    2FA would also make it much harder to use the "somebody hacked me" defense.
    - Operyl 7y ago
      No at that point they'd be saying "zomg NPM has a terrible bug that allowed the hacker to bypass my 2FA!" If someone is trying to redirect blame they'll always find some way to do so.
      - JulianMorrison 7y ago
        The problem is not liars, the problem is being unable to prove they were lying. With 2FA, the logs would prove it.
        Operyl 7y ago
        Unfortunately, based on preexisting cases it really doesn't make much of a difference, the liars will still deflect the blame and the logs will always be "wrong".
  - btown 7y ago
    To quote the article: "As far as we are aware, the only purpose of the malicious code was to sabotage the purescript npm installer to prevent it from running successfully... the purpose of this condition [in the code, hardcoded to include the word 'cli'] seems to be to ensure that the malicious code only runs when our installer is being used (and not @shinnn’s)." :hmm:
ga-vu 7y ago
Not actually malicious. It doesn't steal user data, drop malware, or damage a computer. Just crashes the library. Looks like another developer-developer slap fight.
- braythwayt 7y ago
  I beg your pardon, but if I am using this library as part of a shipping piece of software-as-a-service, and I am in the middle of shipping a new feature when suddenly things mysteriously crash... If I later discover that the crash was put there deliberately, I am going to call that malice, and malice that has directly impacted a functioning business and its customers. It's no different than a disgruntled person putting tacks on the road outside of a supplier. If my truck goes there, gets a flat, and crashes into the ditch as a result, I would call that malice as well. Deliberately crashing software that other people depend upon is malice.
  - johnchristopher 7y ago
    > If I later discover that the crash was put there deliberately, I am going to call that malice, and malice that has directly impacted a functioning business and its customers. Hey, it's open source party ! Where is your patch ? /s edit: oh, bummer. Not really open source. Still: THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, etc. How are we supposed to have code free of malice in the current javascript ecosystem ?
    - luckylion 7y ago
      You touch on an interesting point there if the typical disclaimer is included: does that actually protect against willfully damaging code? It's certainly (and reasonably so) very hard to hold a developer accountable for a bug in an open source library they published. But what if they put rm -rf / in the installer with the intent to delete the files of anybody running it? Does "you should've looked" really work there, or is there an assumption of good nature and malicious behavior voids the disclaimer?
hombre_fatal 7y ago
Another reminder of how annoying it is for a package system to have unqualified package names. Having to ask someone to gift a `purescript` package shouldn't even be a thing. It should've been `@shinn/purescript` and the compiler developers just create their own `@whatever/purescript`. This is something Elm and many others got right. https://package.elm-lang.org/ https://package.elm-lang.org/ It's just infinitely, obviously better. You see all sorts of problems because of this, like people "giving packages away" when they quit. Or buying package names. Or coming up with annoying name hacks because the obvious, best name is simply taken. Or people thinking/guessing that `npm install mysql` is the correct/best/canonical package because it's the simplest name, and anyone who publishes a better library has to name it mysql2 or better-mysql, etc. These just shouldn't even be things.
- McGlockenshire 7y ago
  Even PHP gets this right. It's not hard. It makes me wonder why npm hasn't already moved to namespaced package names.
  - Fellshard 7y ago
    Mostly because the vast majority of JS developers don't seem to be aware of the rest of the software universe, and so seem to reinvent the wheel, rediscover the worst of software's history, and discard the most useful of software findings with shocking regularity. NPM tends to reinforce the worst of the JS world's tendencies.
    - geofft 7y ago
      Perl (cpan), Python (pip or conda), Ruby (gem), and Rust (cargo) all behave as NPM does, so that doesn't seem to be the explanation here.
      - twic 7y ago
        cargo, at least, does this for the reason Fellshard gives - the Rust team essentially commissioned a copy of Ruby's bundler, without considering whether there was anything to learn from any other language's ecosystem.
deleted 7y ago
[deleted]
sieabahlpark 7y ago
Well I guess it's good I didn't start using this after an article I saw yesterday about it.
- ff_ 7y ago
  The facts the article refers to happened two weeks ago and have since been resolved, taking further steps to reduce the chance of this happening again in the future (e.g. by vendoring a lot of code)
  - sieabahlpark 7y ago
    Well that's good
z3t4 7y ago
Before "tree shaking" I stored all npm modules in SCM and reviwed all updates as I had to commit after "npm update". I also put ton of files in .ignore as 90% of files in some packages are not required. I also used to include npm modules in distribution/deployment. So my request to npm is to add an option in the main package.json to disable tree shaking.
- kreetx 7y ago
  I evaluated wordpress plugins this way, a good way to check if something strange had crept in. By the ignored files do you mean non-code?
  - z3t4 7y ago
    Take the most popular npm module lodash for example. It has over one thousand files! But you probably only need one (lodash.js) and that's the one I would commit to SCM.
  - aspaceman 7y ago
    I think they mean code - just remove all files that your program doesn't touch. With something like JS, this should be doable in cases because of the way libraries tend to be designed. The equivalent in a compiled language would be to strip unused symbols from the binary.
- robocat 7y ago
  I wish there was a way to "bless" packages when they were reviewed. I want a network of trust, such that a Google reviewed package is worth 10 points, a package fuzzed by foobar is worth 2 points, something skimmed by a dependant user is worth 1 point etc. I can then chose a compromise between a highly rated/reviewed dependencies and functionality/risk/cost-to-review. My own blessing of a package I have reviewed might become a very small signal in a web of trust.
  - MaulingMonkey 7y ago
    For Rust there is at least one tool for this that I've started using, although I don't think it's very widely used: https://github.com/dpc/crev/tree/master/cargo-crev https://github.com/dpc/crev/tree/master/cargo-crev Ironically this depends on a whole slew of additional packages I haven't yet reviewed to build ;) Very recently (2 commits starting 8 days ago) someone started on an npm equivalent but it's not in a usable state yet: https://github.com/kspaans/npm-crev/blob/master/TODO.md https://github.com/kspaans/npm-crev/blob/master/TODO.md Two problems - few people out there reviewing code, and poor discoverability for who else is reviewing code that you might trust at the moment.
nightkoder 7y ago
I have been installing purescript using Nix from https://github.com/justinwoo/easy-purescript-nix https://github.com/justinwoo/easy-purescript-nix for a while now. It works quite well and I get to avoid npm.
- Shoue 7y ago
  Purescript (purs) is in unstable now as well
SanchoPanda 7y ago
This is not the first time this year we see an npm issue, and it could have been much worse than this. All package managers in general create risks, but how the community etiquette evolves around package managers is just as important. Something is wrong with the latter here.
ruuda 7y ago
One of the things I like about Purescript is that you can use it without needing any javascript package manager, and without running any javascript outside of a browser. Nix works well for installing the Purescript compiler as well as psc-package. https://nixos.org/nixos/packages.html#purescript https://nixos.org/nixos/packages.html#purescript
quickthrower2 7y ago
Part of the problem is the bounty for attacking NPM packages is high. You get a high profile exploit and lots of people talking about it, or you can even get some of your evil JS code running on thousands of sites on the back end or the front end. Compounded by the fact there is no decent base class library for JS like you'd get for .NET [0]. Want to do anything you could do by default with .NET BCL? Like open a url, save a file (with nice api) or parse some XML? Then npm i ... it is. And hope it doesn't pull in an exploit. As a mitigation I recommend people consider writing their own code (NIH) for simple stuff not npm i all the things. [0] I'm comparing to .NET but same could be said of Java/Python/Ruby etc.
Benjammer 7y ago
I always think of this article when these things come up: https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://hackernoon.com/im-harvesting-credit-card-numbers-and...
- lyian 7y ago
  Holy. Good Story...
throwaway3627 7y ago
You know a platform doesn't care about security if either: a. They don't do end-to-end integrity and non-repudiation (not signed hashes of files, not just https, not just hashes, but signed archives/files that can be verified as coming from the developer either with gpg, s/mime or x509 certs) b. They allow packages to execute code or scripts on download or installation And, they don't care about your time if they don't automatically offer a prebuilt, reproducible binary mechanism with a build-from-source install/verification option.
deleted 7y ago
[deleted]
dmix 7y ago
This was first posted 16 days ago https://news.ycombinator.com/from?site=garrood.me https://news.ycombinator.com/from?site=garrood.me
gitgud 7y ago
NPM gets a lot of hate for it's dependency managemnet, but I'm not sure what a solution would be to this problem. - They can't currate packages, or else that friction will drastically slow down the ecosystem (1000's of packages get published everyday). - They can't remove/disable packages (most of the time), or dependencies will no longer be strictly immutable. - They can't disable sub-dependencies, or else this would greatly reduce code reuse and increase redundancy and complexity of packages (every package may have to roll there own X, or compile their package dependencies into bundled JS with no dependencies). I think the problem is simply; it's a low friction dependency management solution -> which made it so popular -> which is making it a target for malicious actors.
- ricardobeat 7y ago
  After 10 years of nodejs I can honestly say I wouldn’t mind the friction. NPM is a wasteland of abandoned packages and reinventing the wheel. They never solved discovery so you have 500x implementations of the exact same thing. Around 2015 we passed the point where looking for the “right” package takes longer than writing your own.
  - xeonoex 7y ago
    See, .NET get a lot of hate from the open source community, but this reason is one of the big ones of why I prefer it. Even when you compare it to Java, I feel like it's way better in this regard. If I want to work with JSON, I use Newtonsoft.JSON. If I want an ORM I'll use Dapper (lightweight) or Entity Framework. So many libraries that would you need if you're using Java or JS is just built in to the standard library. I know it's not a 100% fair comparison, since I am biased. Part of it also might be because .NET is younger and less widely used, but I feel it's pretty true. Though, Python has ton of open source support and is better in this regard.
    - filleduchaos 7y ago
      > I know it's not a 100% fair comparison, since I am biased. It might also have something to do with you apparently unironically comparing a framework to a programming language.
a-dub 7y ago
I wouldn't use the words "malicious" or "exploit" wrt this... It's more like, I dunno, trolling on planet JavaScript? I feel like there should be a big Twitter fight about it...
- Thorrez 7y ago
  shinnn is claiming his account was hacked, and the hacker added the code. Generally if a hacker hacks someone's npm account and adds harmful code, that would be considered malicious and an exploit.
  - Kiro 7y ago
    That's obviously a lie though.
    - Thorrez 7y ago
      Yes, but it sounds like Harry Garrood doesn't have enough evidence to outright accuse shinnn of lying, and the NPM team also is acting as if shinnn isn't lying. So instead of going on a lone and not-fully-backed-up crusade accusing shinnn of lying that could wind up in a he-said-she-said situation with negative fallout, Harry instead decided to use shinnn's words for his own benefit, and hype it up as a serious NPM account hacker inserting malicious code.
- nameiscubanpete 7y ago
  That was my first thought. But then I realized some guy basically broke something so his stuff would work and someone else's wouldn't. He didn't destroy files, but that was malicious as hell.
  - a-dub 7y ago
    mean spirited and dramatic as hell, yes... also, a bad place where real "malicious" things could be done. but "malicious" has a specific meaning and this didn't affect users. more like dramaticious if you ask me... but also uncovers actual dangerous weaknesses in the npm delivery pipeline...
weq 7y ago
So what you are saying, is all the code i see on blogs, demoing that cool little JS thingy, is actually just demo code, not prod ready. To get prod ready, u need to break the NPM dep, and vet everything on your own. So your telling me npm run serve isnt good enough for prod either?! Lies, more lies and broken promises!
s_Hogg 7y ago
High-velocity ecosystems like this make it way too easy to optimise your code for getting pats on the head. It's good that people are pointing out things like this (relatively speaking) not long after the fact, and that a lot of people on here are annoyed about it. The fix is cultural as much as any 2FA implementation.
austincheney 7y ago
Uniqueness and trust of package naming is easily solved: use a URI. It doesn't even have to resolve. URI is a naming convention that invokes both uniqueness and universality.
blackoil 7y ago
What JS needs a standard library(s). Developed and manintained by MS/Google or preferably a foundation. No reason every library have a 10 level deep dependencies.
runeks 7y ago
The work “exploit” is used several times but none of the code seems to exploit anything. Also, “malicious code” usually has a different meaning than something that intentionally makes the program crash during the installation process.
- rjmunro 7y ago
  I feel like this is only part of a wider attack - like by causing this not to download, it meant that users do some other action which opens them up to the real attack.
  - jeltz 7y ago
    Nah, it was probably a childish old maintainer who decided to sabotage it.
- samirm 7y ago
  Agreed about this not being an exploit, but this is definitely malicious code.

521 points by braythwayt 7y ago

The real issue the Balkanization of JavaScript programs. The `rate-map` package is essentially one line of code: start + val * (end - start); https://github.com/shinnn/rate-map/blob/90c234c9/index.mjs#L38 https://github.com/shinnn/rate-map/blob/90c234c9/index.mjs#L...

SCLeo 7y ago
I honestly don't understand why people use packages like this. If I need this functionality, I will simply write my own. Plus, I will never able to find this specific package. I guess PureScript uses this because its author is also the author of rate-map.
- ricardobeat 7y ago
  I’ll give you one: it’s code already written and tested by > 1 person, edge cases already figured out. Saves you time. The gains are small but quickly add up. This is why lately I’ve been a fan of very extensive standard libraries (like Crystal has) - its like having a huge repository but vetoed by the same team and without any of the package management drawbacks.
  - spenczar5 7y ago
    > The gains are small but quickly add up. There are costs to these micro-libraries that outweigh the gains. This code is trivial; there aren’t really edge cases to be worked out.
    - jasonhansel 7y ago
      Also: the way the library handles those edge cases isn't necessarily the way you want. Case in point: rate-map throws exceptions in situations where you might expect it to fail more gracefully.
  - cannedslime 7y ago
    What gains? That line wouldn't take much resources or time to figure out and test. Now you have yet another dependency that could be injected with bad code in the future...
  - vhakulinen 7y ago
    > it’s code already written and tested by > 1 person, edge cases already figured out. It is tested? Edge cases figured out? Essentially for this code?: return start + val * (end - start); Sure, if its running in hostile environment, you might need to do those sanity checks for your parameters - but I have hard time imagining such situation. If you actually need to "Map a number in the range of 0-1 to a new value with a given range" in your own code, can't you guarantee that the variables are all numbers? Its your responsibility as a developer to know your code, and the data your code is handling. > Saves you time. There is something really wrong if finding a package to do this niche thing is faster and more optimal that just writing out that one line of code.
  - inimino 7y ago
    [C]ode already written and tested by > 1 person, edge cases already figured out wastes time when that person was not you. This costs you time. The losses are small but they add up.
  - tjpnz 7y ago
    >The gains are small but quickly add up. Same could also be said for build times not to mention security issues should one of your micro-dependencies be hijacked.
  - ben509 7y ago
    > edge cases already figured out. Nope, it doesn't guarantee its invariants. This would pass all the tests and yet returns a value out of bounds: > s = 1e12 1000000000000 > e = 1e-8 1e-8 > s + 1.0 * (e - s) 0
  - dangoor 7y ago
    > This is why lately I’ve been a fan of very extensive standard libraries (like Crystal has) - its like having a huge repository but vetoed by the same team and without any of the package management drawbacks. Having had some involvement in the early days of node, I had imagined there being something like the Python stdlib. When the npm world grew, I thought "oh, that's pretty cool. It's neat how npm can handle multiple versions of the same package." Now, I'm in absolute agreement with you. There are definitely downsides to a large standard library, but I think the upsides are worth it if that library is maintained.
- dmitriid 7y ago
  Funnily enough, in my opinion, the JS ecosystem fully embraced the Unix way: have small programs/libs that do only one thing and do it (somewhat) well.
  - lenkite 7y ago
    I fear your analogy does not go far enough if you are comparing the JS ecosystem to the UNIX philosophy. Javascript would offer a library for every single option, variant and logical operator for a UNIX command. Combinatorial explosion will devour the web. It already destroys dev laptops anyways when downloading something via NPM.
  - inimino 7y ago
    No, they do not on the whole do anything well.
- GrumpyNl 7y ago
  You should take a look at the trim package and how often hat is downloaded.
  - reeeeee 7y ago
    There is not even a link to the source code on the npm page for it. I installed it and inspected the source code, but I doubt everyone does this when installing a dependency.
    - delfaras 7y ago
      Here's the complete source for anyone curious ``` exports = module.exports = trim; function trim(str){ return str.replace(/^\s|\s$/g, ''); } exports.left = function(str){ return str.replace(/^\s/, ''); }; exports.right = function(str){ return str.replace(/\s$/, ''); }; ```
      - rjmunro 7y ago
        That code doesn't match what actually happens. It will only trim a single character. Have asterisks been trimmed in your copy-paste or something?
        delfaras 7y ago
        Ah you're right, the asterisks have been interpreted as italics. Can't edit my comment, sorry Here's a paste: https://pastebin.com/kBHprdyj https://pastebin.com/kBHprdyj
- sergiomattei 7y ago
  As usual, everyone here will start their "I'm better than this" comments. If you've ever used a code dependency, you are a target for malicious code. That's just how it is. In this case, using small packages like this helps in... 1. Reliability - these packages are typically 100% unit tested 2. Convenience 3. Reduce codebase size. I can't imagine having to copy paste every little small function into a mega utils file.
jasonhansel 7y ago
...wow. I literally did not believe that until I clicked the link. JavaScript has gone too far.
- iamleppert 7y ago
  The problem is clearly due to vanity metrics like number of packages motivating people to publish an insane number of useless packages to fluff their contributions.
  - inimino 7y ago
    That is not clear.
  - ivanfon 7y ago
    Agreed - I’ve found and been astonished by Github users who maintain hundreds of these little npm packages, all of which have usually under 10 lines of actual code, and sometimes even have chains of dependencies on the user’s other packages. It seems like the only reason these packages get any significant downloads is when one of them gets depended on by a big package, causing the entire dependency chain of the user’s little packages to be downloaded.
- fock 7y ago
  well, the rate-map at least includes a function: https://github.com/sindresorhus/shebang-regex https://github.com/sindresorhus/shebang-regex
  - hu3 7y ago
    A dependency for /^#!(.*)/; With 8.6 mil downloads and 71 dependents... https://www.npmjs.com/package/shebang-regex https://www.npmjs.com/package/shebang-regex
    - fock 7y ago
      and if you look closely, 70 of those dependents are just small-scale/bs packages and the only sensible real dependency is cross-spawn _via_ the shebang package (which applies this regex to a string). The whole ecosysten could use a purge of all those useless 1-line-requires (which were all introduced by helpful commits from the "i has 1337 downloads"-community), currently this is madness.
ww520 7y ago
But it's great for resume and portfolio.
- s_Hogg 7y ago
  This is the problem - how to disincentivise that behaviour?
  - throwaway66666 7y ago
    Do not hire people who do this. Most of them will say "over 200m downloads of npm packages". Go and take a look. If you see things like 'is-not-foo, checks if a given string is not equal to the string "foo"', and 0 meaningful contributions, either pay no attention to these claims or pass.
tonetheman 7y ago
The real issue is most JS programmers are not smart enough to write it so they have to find packages to do anything.
- dang 7y ago
  Hey, can you please stop breaking the site guidelines? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html I don't want to ban you, but when accounts keep doing this, we kind of have to.
na85 7y ago
It's an ecosystem full of reinventing the wheel. The most popular library, lodash, includes a reimplementation of a foreach loop for Pete's sake, for reasons passing understanding since it's part of the ecma spec. JavaScript is just amateur hour, and these things are going to keep happening. It's pathological.
- rwbcxrz 7y ago
  Which foreach are you talking about? Array.prototype.forEach, for...in loops, for...of loops? The first only works with arrays and array-like objects. The second works on objects and arrays, but it iterates over all enumerable properties, so you don't want really want to use it for arrays. It's also made a lot less useful because it only iterates over properties, not keys. The third finally provides some sanity, but it's only been around since ES6. Before that, lodash's each method was the most reliable way to iterate over a collection, be it an object or an array. Just because you don't know the reason for something doesn't mean there isn't one.
  - runarberg 7y ago
    `for ... of` only iterate over objects that implement `Symbol.iterator`. Native objects don’t do that by default, so `_.forEach` is more useful the `for ... of` even if you are only targeting modern browsers and not compiling the code down to an earlier version of the spec. That said, you can use `Object.keys`, `Object.values`, or `Object.entries` if you want to iterate over objects that don’t implement `Symbol.iterator`, so if you only need `_.forEach` there is no reason to pull in any libraries. Object.entries(object).forEach(([key, value]) => { // ... });
  - inimino 7y ago
    > Array.prototype.forEach Yes, this one. If the object is an Array. According to whatever test they are using for that. Lodash includes a reimplementation of Array.prototype.forEach because mistakes were made. It also works on other objects because other mistakes were made. We all make mistakes. But just because there is a reason for something does not mean there is a good reason.
aur09 7y ago
Wow—and spread into 11 files to facilitate CI, linting, etc—and with 1kb worth of error checking. That is nuts!
TeMPOraL 7y ago
And this one-liner library even has a dependency (!) used by the surrounding error-checking code.
- ddalex 7y ago
  What's the difference between an NPM dependency chain and a black hole? We can measure the depth of a black hole.
lenkite 7y ago
Don't know whether to laugh or cry at this. Truth is stranger than fiction in the land of JavaScript. Could any developer 20 years ago predict this is what software engineering on the Web would devolve to ?

bsamuels 7y ago

I wonder how bad will this cred-stuffing package authors problem will get before npm/other package managers flat out require 2FA for maintainers

hn_throwaway_99 7y ago
I think the blog author is implying as much as he can, without directly accusing, that he believes that https://github.com/shinnn https://github.com/shinnn was responsible for the bad code, not a random hack.
- JulianMorrison 7y ago
  2FA would also make it much harder to use the "somebody hacked me" defense.
  - Operyl 7y ago
    No at that point they'd be saying "zomg NPM has a terrible bug that allowed the hacker to bypass my 2FA!" If someone is trying to redirect blame they'll always find some way to do so.
    - JulianMorrison 7y ago
      The problem is not liars, the problem is being unable to prove they were lying. With 2FA, the logs would prove it.
      - Operyl 7y ago
        Unfortunately, based on preexisting cases it really doesn't make much of a difference, the liars will still deflect the blame and the logs will always be "wrong".
- btown 7y ago
  To quote the article: "As far as we are aware, the only purpose of the malicious code was to sabotage the purescript npm installer to prevent it from running successfully... the purpose of this condition [in the code, hardcoded to include the word 'cli'] seems to be to ensure that the malicious code only runs when our installer is being used (and not @shinnn’s)." :hmm:

ga-vu 7y ago

Not actually malicious. It doesn't steal user data, drop malware, or damage a computer. Just crashes the library. Looks like another developer-developer slap fight.

braythwayt 7y ago
I beg your pardon, but if I am using this library as part of a shipping piece of software-as-a-service, and I am in the middle of shipping a new feature when suddenly things mysteriously crash... If I later discover that the crash was put there deliberately, I am going to call that malice, and malice that has directly impacted a functioning business and its customers. It's no different than a disgruntled person putting tacks on the road outside of a supplier. If my truck goes there, gets a flat, and crashes into the ditch as a result, I would call that malice as well. Deliberately crashing software that other people depend upon is malice.
- johnchristopher 7y ago
  > If I later discover that the crash was put there deliberately, I am going to call that malice, and malice that has directly impacted a functioning business and its customers. Hey, it's open source party ! Where is your patch ? /s edit: oh, bummer. Not really open source. Still: THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, etc. How are we supposed to have code free of malice in the current javascript ecosystem ?
  - luckylion 7y ago
    You touch on an interesting point there if the typical disclaimer is included: does that actually protect against willfully damaging code? It's certainly (and reasonably so) very hard to hold a developer accountable for a bug in an open source library they published. But what if they put rm -rf / in the installer with the intent to delete the files of anybody running it? Does "you should've looked" really work there, or is there an assumption of good nature and malicious behavior voids the disclaimer?

hombre_fatal 7y ago

Another reminder of how annoying it is for a package system to have unqualified package names. Having to ask someone to gift a `purescript` package shouldn't even be a thing. It should've been `@shinn/purescript` and the compiler developers just create their own `@whatever/purescript`. This is something Elm and many others got right. https://package.elm-lang.org/ https://package.elm-lang.org/ It's just infinitely, obviously better. You see all sorts of problems because of this, like people "giving packages away" when they quit. Or buying package names. Or coming up with annoying name hacks because the obvious, best name is simply taken. Or people thinking/guessing that `npm install mysql` is the correct/best/canonical package because it's the simplest name, and anyone who publishes a better library has to name it mysql2 or better-mysql, etc. These just shouldn't even be things.

McGlockenshire 7y ago
Even PHP gets this right. It's not hard. It makes me wonder why npm hasn't already moved to namespaced package names.
- Fellshard 7y ago
  Mostly because the vast majority of JS developers don't seem to be aware of the rest of the software universe, and so seem to reinvent the wheel, rediscover the worst of software's history, and discard the most useful of software findings with shocking regularity. NPM tends to reinforce the worst of the JS world's tendencies.
  - geofft 7y ago
    Perl (cpan), Python (pip or conda), Ruby (gem), and Rust (cargo) all behave as NPM does, so that doesn't seem to be the explanation here.
    - twic 7y ago
      cargo, at least, does this for the reason Fellshard gives - the Rust team essentially commissioned a copy of Ruby's bundler, without considering whether there was anything to learn from any other language's ecosystem.

deleted 7y ago

[deleted]

sieabahlpark 7y ago

Well I guess it's good I didn't start using this after an article I saw yesterday about it.

ff_ 7y ago
The facts the article refers to happened two weeks ago and have since been resolved, taking further steps to reduce the chance of this happening again in the future (e.g. by vendoring a lot of code)
- sieabahlpark 7y ago
  Well that's good

z3t4 7y ago

Before "tree shaking" I stored all npm modules in SCM and reviwed all updates as I had to commit after "npm update". I also put ton of files in .ignore as 90% of files in some packages are not required. I also used to include npm modules in distribution/deployment. So my request to npm is to add an option in the main package.json to disable tree shaking.

kreetx 7y ago
I evaluated wordpress plugins this way, a good way to check if something strange had crept in. By the ignored files do you mean non-code?
- z3t4 7y ago
  Take the most popular npm module lodash for example. It has over one thousand files! But you probably only need one (lodash.js) and that's the one I would commit to SCM.
- aspaceman 7y ago
  I think they mean code - just remove all files that your program doesn't touch. With something like JS, this should be doable in cases because of the way libraries tend to be designed. The equivalent in a compiled language would be to strip unused symbols from the binary.
robocat 7y ago
I wish there was a way to "bless" packages when they were reviewed. I want a network of trust, such that a Google reviewed package is worth 10 points, a package fuzzed by foobar is worth 2 points, something skimmed by a dependant user is worth 1 point etc. I can then chose a compromise between a highly rated/reviewed dependencies and functionality/risk/cost-to-review. My own blessing of a package I have reviewed might become a very small signal in a web of trust.
- MaulingMonkey 7y ago
  For Rust there is at least one tool for this that I've started using, although I don't think it's very widely used: https://github.com/dpc/crev/tree/master/cargo-crev https://github.com/dpc/crev/tree/master/cargo-crev Ironically this depends on a whole slew of additional packages I haven't yet reviewed to build ;) Very recently (2 commits starting 8 days ago) someone started on an npm equivalent but it's not in a usable state yet: https://github.com/kspaans/npm-crev/blob/master/TODO.md https://github.com/kspaans/npm-crev/blob/master/TODO.md Two problems - few people out there reviewing code, and poor discoverability for who else is reviewing code that you might trust at the moment.

nightkoder 7y ago

I have been installing purescript using Nix from https://github.com/justinwoo/easy-purescript-nix https://github.com/justinwoo/easy-purescript-nix for a while now. It works quite well and I get to avoid npm.

Shoue 7y ago
Purescript (purs) is in unstable now as well

SanchoPanda 7y ago

This is not the first time this year we see an npm issue, and it could have been much worse than this. All package managers in general create risks, but how the community etiquette evolves around package managers is just as important. Something is wrong with the latter here.

ruuda 7y ago

One of the things I like about Purescript is that you can use it without needing any javascript package manager, and without running any javascript outside of a browser. Nix works well for installing the Purescript compiler as well as psc-package. https://nixos.org/nixos/packages.html#purescript https://nixos.org/nixos/packages.html#purescript

quickthrower2 7y ago

Part of the problem is the bounty for attacking NPM packages is high. You get a high profile exploit and lots of people talking about it, or you can even get some of your evil JS code running on thousands of sites on the back end or the front end. Compounded by the fact there is no decent base class library for JS like you'd get for .NET [0]. Want to do anything you could do by default with .NET BCL? Like open a url, save a file (with nice api) or parse some XML? Then npm i ... it is. And hope it doesn't pull in an exploit. As a mitigation I recommend people consider writing their own code (NIH) for simple stuff not npm i all the things. [0] I'm comparing to .NET but same could be said of Java/Python/Ruby etc.

Benjammer 7y ago

I always think of this article when these things come up: https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://hackernoon.com/im-harvesting-credit-card-numbers-and...

lyian 7y ago
Holy. Good Story...

throwaway3627 7y ago

You know a platform doesn't care about security if either: a. They don't do end-to-end integrity and non-repudiation (not signed hashes of files, not just https, not just hashes, but signed archives/files that can be verified as coming from the developer either with gpg, s/mime or x509 certs) b. They allow packages to execute code or scripts on download or installation And, they don't care about your time if they don't automatically offer a prebuilt, reproducible binary mechanism with a build-from-source install/verification option.

deleted 7y ago

[deleted]

dmix 7y ago

This was first posted 16 days ago https://news.ycombinator.com/from?site=garrood.me https://news.ycombinator.com/from?site=garrood.me

gitgud 7y ago

NPM gets a lot of hate for it's dependency managemnet, but I'm not sure what a solution would be to this problem. - They can't currate packages, or else that friction will drastically slow down the ecosystem (1000's of packages get published everyday). - They can't remove/disable packages (most of the time), or dependencies will no longer be strictly immutable. - They can't disable sub-dependencies, or else this would greatly reduce code reuse and increase redundancy and complexity of packages (every package may have to roll there own X, or compile their package dependencies into bundled JS with no dependencies). I think the problem is simply; it's a low friction dependency management solution -> which made it so popular -> which is making it a target for malicious actors.

ricardobeat 7y ago
After 10 years of nodejs I can honestly say I wouldn’t mind the friction. NPM is a wasteland of abandoned packages and reinventing the wheel. They never solved discovery so you have 500x implementations of the exact same thing. Around 2015 we passed the point where looking for the “right” package takes longer than writing your own.
- xeonoex 7y ago
  See, .NET get a lot of hate from the open source community, but this reason is one of the big ones of why I prefer it. Even when you compare it to Java, I feel like it's way better in this regard. If I want to work with JSON, I use Newtonsoft.JSON. If I want an ORM I'll use Dapper (lightweight) or Entity Framework. So many libraries that would you need if you're using Java or JS is just built in to the standard library. I know it's not a 100% fair comparison, since I am biased. Part of it also might be because .NET is younger and less widely used, but I feel it's pretty true. Though, Python has ton of open source support and is better in this regard.
  - filleduchaos 7y ago
    > I know it's not a 100% fair comparison, since I am biased. It might also have something to do with you apparently unironically comparing a framework to a programming language.

a-dub 7y ago

I wouldn't use the words "malicious" or "exploit" wrt this... It's more like, I dunno, trolling on planet JavaScript? I feel like there should be a big Twitter fight about it...

Thorrez 7y ago
shinnn is claiming his account was hacked, and the hacker added the code. Generally if a hacker hacks someone's npm account and adds harmful code, that would be considered malicious and an exploit.
- Kiro 7y ago
  That's obviously a lie though.
  - Thorrez 7y ago
    Yes, but it sounds like Harry Garrood doesn't have enough evidence to outright accuse shinnn of lying, and the NPM team also is acting as if shinnn isn't lying. So instead of going on a lone and not-fully-backed-up crusade accusing shinnn of lying that could wind up in a he-said-she-said situation with negative fallout, Harry instead decided to use shinnn's words for his own benefit, and hype it up as a serious NPM account hacker inserting malicious code.
nameiscubanpete 7y ago
That was my first thought. But then I realized some guy basically broke something so his stuff would work and someone else's wouldn't. He didn't destroy files, but that was malicious as hell.
- a-dub 7y ago
  mean spirited and dramatic as hell, yes... also, a bad place where real "malicious" things could be done. but "malicious" has a specific meaning and this didn't affect users. more like dramaticious if you ask me... but also uncovers actual dangerous weaknesses in the npm delivery pipeline...

weq 7y ago

So what you are saying, is all the code i see on blogs, demoing that cool little JS thingy, is actually just demo code, not prod ready. To get prod ready, u need to break the NPM dep, and vet everything on your own. So your telling me npm run serve isnt good enough for prod either?! Lies, more lies and broken promises!

s_Hogg 7y ago

High-velocity ecosystems like this make it way too easy to optimise your code for getting pats on the head. It's good that people are pointing out things like this (relatively speaking) not long after the fact, and that a lot of people on here are annoyed about it. The fix is cultural as much as any 2FA implementation.

austincheney 7y ago

Uniqueness and trust of package naming is easily solved: use a URI. It doesn't even have to resolve. URI is a naming convention that invokes both uniqueness and universality.

blackoil 7y ago

What JS needs a standard library(s). Developed and manintained by MS/Google or preferably a foundation. No reason every library have a 10 level deep dependencies.

runeks 7y ago

The work “exploit” is used several times but none of the code seems to exploit anything. Also, “malicious code” usually has a different meaning than something that intentionally makes the program crash during the installation process.

rjmunro 7y ago
I feel like this is only part of a wider attack - like by causing this not to download, it meant that users do some other action which opens them up to the real attack.
- jeltz 7y ago
  Nah, it was probably a childish old maintainer who decided to sabotage it.
samirm 7y ago
Agreed about this not being an exploit, but this is definitely malicious code.