5 ms·

The npm team's lack of responsible action here is astonishing, if not surprising. If a token for these very popular packages was compromised, then every user of

The npm team's lack of responsible action here is astonishing, if not surprising. If a token for these very popular packages was compromised, then every user of those packages could conceivably have had their tokens compromised, not just those that have it as a downstream dependency. That means every single package published between the vulnerable packages and the mass token invalidation is suspect and should be unpublished until it can be audited. npm's position that "a very small number" of packages were compromised is minimizing a potentially massive issue. Their promise to "[conduct] a deep audit of all the packages in the Registry" both misses the point and is ineffective. I have come to expect no less from npm, and by extension, nodejs.

Prefinem 8y ago
NPM revoked all tokens. How is that irresponsible? https://status.npmjs.org/incidents/dn7c1fgrr7ng https://status.npmjs.org/incidents/dn7c1fgrr7ng
- Benjammer 8y ago
  A malicious change could have already been published somewhere else, couldn't it have? And we just haven't found it yet?
  - Prefinem 8y ago
    Just saying they didn't do nothing. Maybe they didn't do as well as they could have, but they did do something.
    - mirashii 8y ago
      Calling it resolved is worse than doing nothing. If they had done nothing, at least people would know that "If I run npm install now, that's bad". Now they've claimed it is resolved, which tells their users "It's okay to start installing things again" when it isn't safe until an audit has been completed.
- mirashii 8y ago
  For one, they've marked the issue as resolved before they've completed any forensic analysis to discover if additional compromised packages were uploaded with stolen credentials during the window in which they weren't revoked. This gives the false sense that its safe to install packages again.
  - throwaway5752 8y ago
    And the fact they don't get this is incredibly disconcerting.
    - mirashii 8y ago
      There's also no systemic fix for this issue. Next week, the same thing could happen again. It's time to enforce two-factor authentication for publishing packages. It's time to publish a roadmap towards package signing and verification. It's time to talk about sandboxing the install scripts to prevent token theft. We're done for the day is so far from sufficient.
      - deleted 8y ago
        [deleted]
      - nanodeath 8y ago
        > package signing and verification You're the first person I've seen mention this, which seems like it should be the first and most obvious line of defense against bad stuff like this. +1
        smt88 8y ago
        People mention most of these steps every time npm fucks up (which is often). Package management is a mostly solved problem, but npm refuses to learn from the 30+ years of experience that Linux and other communities have had.
        eropple 8y ago
- thenewwazoo 8y ago
  * eslint gets compromised, and 3.7.2 is published * eslint user FooCorp also gets compromised, and a similarly-malicious version of foolib-js gets published that includes the _same code_ to steal tokens * npm invalidates all tokens * you decide to use foolib-js, and your newly-minted token is now compromised npm are fucking this up, and royally.
  - thephyber 8y ago
    While this is possible, I'm willing to give the NPM team at least a little benefit of the doubt that they actually researched the access logs before they state this: > We determined that access tokens for approximately 4,500 accounts could have been obtained before we acted to close this vulnerability. However, we have not found evidence that any tokens were actually obtained or used to access any npmjs.com account during this window.[1] I get that it's possible that other modules could already be infected, but it's also true that other modules could have been similarly infected long before this one. [1] https://blog.npmjs.org/post/175824896885/incident-report-npm-inc-operations-incident-of https://blog.npmjs.org/post/175824896885/incident-report-npm...
    - thenewwazoo 8y ago
      Your quote wonderfully illustrates that npm are either being obfuscatory or entirely missing the point. How did they determine tokens for 4,500 accounts could have been obtained, and what is that even supposed to mean? The problem here is that any user of these packages could have had their .npmjs file read and exfiltrated, not just some upstream package maintainer. Were there only 4,500 valid npm tokens or something? I cannot imagine that is the case. So either they looked at 4,500 packages uploaded during the compromise window and they're not explaining how they undertook to do that, or they don't understand the vector and are minimizing the severity of the issue.
      - w0rd-driven 8y ago
        I would assume their logs would possibly tell them which tokens were associated with the users that downloaded v3.7.2. npm probably doesn't need credentials to download a package so the number of downloads is likely higher. Determining other packages affected are another matter entirely and no one can say this attack vector is only bound by this specific date window. This could've been way more widespread unless they're unpacking payloads and grepping for key pieces of this specific attack. I think it would be helpful if they could expose some of those logs but considering the meat of what matters would be the IP addresses to verify if your machine was compromised (or your CI server) that GPDR effectively wiped that possibility off the table. It would almost behoove them to setup a kind of haveibeenpwned service where you can check against stuff like this in the future. It's not like this can't happen again as the hole hasn't been closed completely, only this one set of compromised packages appears clean for now.
_bxg1 8y ago
"npm has revoked all access tokens issued before 2018-07-12 12:30 UTC. As a result, all access tokens compromised by this attack should no longer be usable."
- thenewwazoo 8y ago
  "access tokens compromised by this attack should no longer be useable", but tokens compromised by other compromised packages published during the intervening period will still compromise you.
_greim_ 8y ago
> That means every single package published between the vulnerable packages and the mass token invalidation is suspect and should be unpublished until it can be audited. I wonder how many packages in fact got published during that nine-hour time window. Does anyone know what kind of number that would be?
- thenewwazoo 8y ago
  I'm currently dumping the npm db right now to figure that out. eslint-scope 3.7.2 was published on 2018-07-12T10:40:00.478Z eslint-config-eslint 5.0.2 was published on 2018-07-12T09:49:24.957Z tokens were invalidated at 2018-07-12 12:30 UTC
  - _greim_ 8y ago
    > tokens were invalidated at 2018-07-12 12:30 UTC I'm reading it as tokens created before 2018-07-12 12:30 UTC were invalidated, but the invalidation itself happened on 2018-07-12 18:42 UTC according to eslint's post-mortem[1]. I think the latter is the more relevant date. [1] https://eslint.org/blog/2018/07/postmortem-for-malicious-package-publishes https://eslint.org/blog/2018/07/postmortem-for-malicious-pac...
    - deleted 8y ago
      [deleted]
- watson 8y ago
  If I understand the npm incident report correctly, the answer is none: > We determined that access tokens for approximately 4,500 accounts could have been obtained before we acted to close this vulnerability. However, we have not found evidence that any tokens were actually obtained or used to access any npmjs.com account during this window. Source: https://blog.npmjs.org/post/175824896885/incident-report-npm-inc-operations-incident-of https://blog.npmjs.org/post/175824896885/incident-report-npm...
  - _greim_ 8y ago
thenewwazoo 8y ago
You can see a list of all npm packages uploaded between the time that the first compromised eslint package was uploaded and the token invalidation time here: https://gist.github.com/thenewwazoo/0306aa06aafe7807497ed1db430ee2b8 https://gist.github.com/thenewwazoo/0306aa06aafe7807497ed1db...
- thenewwazoo 8y ago
  To follow up, I've downloaded all of the versions indicated here and none include the strings `raw/XLeVP82h`, `sstatic1.histats.com`, nor `statcounter`, and none of the included `build.js` files contain `eval`. Not an exhaustive test.
- thenewwazoo 8y ago
  Further follow up: I've updated the gist to include all packages between the time at which the first compromised package was uploaded and the time at which the keys were invalidated (which was later than the invalidation threshold). I'm working on auditing the larger (~2000) list of packages now.
- thenewwazoo 8y ago
  Final follow-up: a more-exhaustive search also returned no evals in build.js files, nor any obviously-suspicious strings.
  - vincnetas 8y ago
    Have you checked for \u0065val or e\u0076al or any other combination of it?
    - sillysaurus3 8y ago
      This is impossible in general. eval === this[[1,18,-3,8].map(x=>String.fromCharCode(x+100)).join("")] https://jsfiddle.net/hkvu9s47/ https://jsfiddle.net/hkvu9s47/
      - 8y ago

Prefinem 8y ago

NPM revoked all tokens. How is that irresponsible? https://status.npmjs.org/incidents/dn7c1fgrr7ng https://status.npmjs.org/incidents/dn7c1fgrr7ng

Benjammer 8y ago
A malicious change could have already been published somewhere else, couldn't it have? And we just haven't found it yet?
- Prefinem 8y ago
  Just saying they didn't do nothing. Maybe they didn't do as well as they could have, but they did do something.
  - mirashii 8y ago
    Calling it resolved is worse than doing nothing. If they had done nothing, at least people would know that "If I run npm install now, that's bad". Now they've claimed it is resolved, which tells their users "It's okay to start installing things again" when it isn't safe until an audit has been completed.
mirashii 8y ago
For one, they've marked the issue as resolved before they've completed any forensic analysis to discover if additional compromised packages were uploaded with stolen credentials during the window in which they weren't revoked. This gives the false sense that its safe to install packages again.
- throwaway5752 8y ago
  And the fact they don't get this is incredibly disconcerting.
  - mirashii 8y ago
    There's also no systemic fix for this issue. Next week, the same thing could happen again. It's time to enforce two-factor authentication for publishing packages. It's time to publish a roadmap towards package signing and verification. It's time to talk about sandboxing the install scripts to prevent token theft. We're done for the day is so far from sufficient.
    - deleted 8y ago
      [deleted]
    - nanodeath 8y ago
      > package signing and verification You're the first person I've seen mention this, which seems like it should be the first and most obvious line of defense against bad stuff like this. +1
      - smt88 8y ago
        People mention most of these steps every time npm fucks up (which is often). Package management is a mostly solved problem, but npm refuses to learn from the 30+ years of experience that Linux and other communities have had.
        eropple 8y ago
thenewwazoo 8y ago
* eslint gets compromised, and 3.7.2 is published * eslint user FooCorp also gets compromised, and a similarly-malicious version of foolib-js gets published that includes the _same code_ to steal tokens * npm invalidates all tokens * you decide to use foolib-js, and your newly-minted token is now compromised npm are fucking this up, and royally.
- thephyber 8y ago
  While this is possible, I'm willing to give the NPM team at least a little benefit of the doubt that they actually researched the access logs before they state this: > We determined that access tokens for approximately 4,500 accounts could have been obtained before we acted to close this vulnerability. However, we have not found evidence that any tokens were actually obtained or used to access any npmjs.com account during this window.[1] I get that it's possible that other modules could already be infected, but it's also true that other modules could have been similarly infected long before this one. [1] https://blog.npmjs.org/post/175824896885/incident-report-npm-inc-operations-incident-of https://blog.npmjs.org/post/175824896885/incident-report-npm...
  - thenewwazoo 8y ago
    Your quote wonderfully illustrates that npm are either being obfuscatory or entirely missing the point. How did they determine tokens for 4,500 accounts could have been obtained, and what is that even supposed to mean? The problem here is that any user of these packages could have had their .npmjs file read and exfiltrated, not just some upstream package maintainer. Were there only 4,500 valid npm tokens or something? I cannot imagine that is the case. So either they looked at 4,500 packages uploaded during the compromise window and they're not explaining how they undertook to do that, or they don't understand the vector and are minimizing the severity of the issue.
    - w0rd-driven 8y ago
      I would assume their logs would possibly tell them which tokens were associated with the users that downloaded v3.7.2. npm probably doesn't need credentials to download a package so the number of downloads is likely higher. Determining other packages affected are another matter entirely and no one can say this attack vector is only bound by this specific date window. This could've been way more widespread unless they're unpacking payloads and grepping for key pieces of this specific attack. I think it would be helpful if they could expose some of those logs but considering the meat of what matters would be the IP addresses to verify if your machine was compromised (or your CI server) that GPDR effectively wiped that possibility off the table. It would almost behoove them to setup a kind of haveibeenpwned service where you can check against stuff like this in the future. It's not like this can't happen again as the hole hasn't been closed completely, only this one set of compromised packages appears clean for now.

_bxg1 8y ago

"npm has revoked all access tokens issued before 2018-07-12 12:30 UTC. As a result, all access tokens compromised by this attack should no longer be usable."

thenewwazoo 8y ago
"access tokens compromised by this attack should no longer be useable", but tokens compromised by other compromised packages published during the intervening period will still compromise you.

_greim_ 8y ago

> That means every single package published between the vulnerable packages and the mass token invalidation is suspect and should be unpublished until it can be audited. I wonder how many packages in fact got published during that nine-hour time window. Does anyone know what kind of number that would be?

thenewwazoo 8y ago
I'm currently dumping the npm db right now to figure that out. eslint-scope 3.7.2 was published on 2018-07-12T10:40:00.478Z eslint-config-eslint 5.0.2 was published on 2018-07-12T09:49:24.957Z tokens were invalidated at 2018-07-12 12:30 UTC
- _greim_ 8y ago
  > tokens were invalidated at 2018-07-12 12:30 UTC I'm reading it as tokens created before 2018-07-12 12:30 UTC were invalidated, but the invalidation itself happened on 2018-07-12 18:42 UTC according to eslint's post-mortem[1]. I think the latter is the more relevant date. [1] https://eslint.org/blog/2018/07/postmortem-for-malicious-package-publishes https://eslint.org/blog/2018/07/postmortem-for-malicious-pac...
  - deleted 8y ago
    [deleted]
watson 8y ago
If I understand the npm incident report correctly, the answer is none: > We determined that access tokens for approximately 4,500 accounts could have been obtained before we acted to close this vulnerability. However, we have not found evidence that any tokens were actually obtained or used to access any npmjs.com account during this window. Source: https://blog.npmjs.org/post/175824896885/incident-report-npm-inc-operations-incident-of https://blog.npmjs.org/post/175824896885/incident-report-npm...
- _greim_ 8y ago

thenewwazoo 8y ago

You can see a list of all npm packages uploaded between the time that the first compromised eslint package was uploaded and the token invalidation time here: https://gist.github.com/thenewwazoo/0306aa06aafe7807497ed1db430ee2b8 https://gist.github.com/thenewwazoo/0306aa06aafe7807497ed1db...

thenewwazoo 8y ago
To follow up, I've downloaded all of the versions indicated here and none include the strings `raw/XLeVP82h`, `sstatic1.histats.com`, nor `statcounter`, and none of the included `build.js` files contain `eval`. Not an exhaustive test.
thenewwazoo 8y ago
Further follow up: I've updated the gist to include all packages between the time at which the first compromised package was uploaded and the time at which the keys were invalidated (which was later than the invalidation threshold). I'm working on auditing the larger (~2000) list of packages now.
thenewwazoo 8y ago
Final follow-up: a more-exhaustive search also returned no evals in build.js files, nor any obviously-suspicious strings.
- vincnetas 8y ago
  Have you checked for \u0065val or e\u0076al or any other combination of it?
  - sillysaurus3 8y ago
    This is impossible in general. eval === this[[1,18,-3,8].map(x=>String.fromCharCode(x+100)).join("")] https://jsfiddle.net/hkvu9s47/ https://jsfiddle.net/hkvu9s47/
    - 8y ago