12 ms·

Npm's Self-Signed Certificate is No More

215 points by rcconf 13y ago

rcconf 13y ago
This has also broken all of our own deploys on Jenkins. We had to use solution 2, and then upgrade, because solution 1 also produces the certificate error. It was surprising that npm status didn't have any sort of warning. I had to find out about this via twitter. I think it's extremely irresponsible and I'm glad we've started to move away from node.js and started to use Java.
- benajnim 13y ago
  That's not a loaded comment.. Changing your programming environment over an SSL certificate? Tell us all about how awesome it is building apps in Java!
  - awj 13y ago
    You want to talk loaded comments? You're reducing "I'm glad to move away from a project that breaks functionality for all users with no notice" to a quibble over an SSL certificate change. That npm as a project thought this was the correct way to handle this kind of transition is very unsettling.
    - 1stop 13y ago
      Yeah, because no one ever had any problems with Maven, that thing is like package heaven, it's all rainbows and unicorns!
      - Pacabel 13y ago
        It's really disappointing to see this sort of "reasoning" used so often these days. Just because somebody points out serious flaws or problems with one technology does not automatically mean that he or she thinks that other, unrelated-yet-similar technologies aren't flawed or are somehow perfect.
        1stop 13y ago
        > I'm glad we've started to move away from node.js and started to use Java. That is what I was responding to directly. The implication from the statement was that npm flaws (i.e certificate changes that break everything) is a good reason to switch to Java. It isn't at all a good reason to switch to java, as java's equivalent would easily waste more time than even this rather embarrassing certificate problem with npm. > unrelated-yet-similar technologies It is related, and similar, it is a like-for-like comparison between nodejs package management and java package management.
        teacup50 13y ago
        > It isn't at all a good reason to switch to java, as java's equivalent would easily waste more time than even this rather embarrassing certificate problem with npm. Works fine for me. Has a healthy ecosystem of 3rd-party artifact repository implementations.
        deleted 13y ago
        [deleted]
        EdwardDiego 13y ago
        > It isn't at all a good reason to switch to java, as java's equivalent would easily waste more time than even this rather embarrassing certificate problem with npm. As someone who has used it daily for the past 5.5 years - not really. That said, I'd prefer something like Gradle, but I can't fault Maven for just working, goddamnit.
      - awj 13y ago
        How is that comparison at all relevant?
        1stop 13y ago
        > I'm glad we've started to move away from node.js and started to use Java. I'm not sure you get how threading works, but this thread originated from the above quote. And comparing npm with Maven given the above statement is relevant, given it is the primary "Package management" system for Java, much like npm.
        awj 13y ago
        I perfectly well understand how threading works. Read the rest of the comment that you cherry picked that quote from, it's about being upset at how npm doesn't seem to be taking it's responsibility seriously. If you had some information about Maven callously performing a user-hostile update, the comparison would be appropriate. As it is you're just relying on "lolz, Java sucks" as a form of argument.
        1stop 13y ago
        No, I'm really not. I don't think java sucks at all, and never said as such. I'm saying Maven isn't better, nor would npm's mistake be a good reason to switch from to Java from Node. npm didn't perform a user-hostile update, they made a mistake with certificate authorities. How many 0-days have forced a java upgrade on people... was that a 'user hostile' move by sun/oracle. I think your line of reasoning is utterly ridiculous, and you are responding to words I didn't say.
  - RyanZAG 13y ago
    He never said he was changing because of this issue. (Logically, if he's already been changing over then it can't have been because of an issue that just happened) "Tell us all about how awesome it is building apps in Java!" sounds like a pretty loaded comment to me though.
  - nolok 13y ago
    > Changing your programming environment over an SSL certificate? That's not at all what he said.
  - twistedpair 13y ago
    In the last 5 years maven (Central) hasn't given me a cert or pgp error. Building 50 times a day in Java is running just fine ;) Of course it could happen in theory to any such similar system such as NuGet if people decided to yank the rug out from under their user base.
- tootie 13y ago
  Ditto. Team wasted a few hours on this today.
ibash 13y ago
As per Joe Grund: http://blog.npmjs.org/post/78085451721/npms-self-signed-certificate-is-no-more#comment-1264542895 http://blog.npmjs.org/post/78085451721/npms-self-signed-cert... npm config set strict-ssl false npm update npm -g npm config set strict-ssl true
- honksillet 13y ago
  I was just about to post this link. The change broke bower at on my laptop. The above link fixed it.
- awj 13y ago
  That's ... still a dangerous way to accomplish the update. You're disabling a key safeguard that ensures you're actually talking to official npm servers, then re-enabling the safeguard after running the code that whoever-you-got-it-from provided. I'm sure the likelihood of experiencing a MITM attack here is low, but the security consideration needs to be as widely advertised as the solution.
atonse 13y ago
I'm surprised that this wasn't one of those "hey in 60 days we'll be making this change and it will break x, y, and z. Be prepared." sort of things.
randunel 13y ago
When package managers misbehave, the whole software bundle suffers. Maybe the Node.JS project should have more (decision) power over its package manager. Node maintainers properly announce api version changes sometimes months in advance, so you don't need to read through the pr/issues on github or mailing lists to stay up to date. I really don't think an advance notice would have hurt. So much for automation.
- agilebyte 13y ago
  Well, nodejitsu are trademarking NPM so let's see how that plays out.
deleted 13y ago
[deleted]
diminoten 13y ago
There's a comment on the blog post that I thought was very good, it was from Rob, and it ends with: "Please, try harder or get forked. Not sure how else to say that."
- johnchristopher 13y ago
  Rob's comment was spot on. Here's a link to the full text: http://blog.npmjs.org/post/78085451721/npms-self-signed-certificate-is-no-more#comment-1264657416 http://blog.npmjs.org/post/78085451721/npms-self-signed-cert...
  - dullroar 13y ago
    And now that comment seems to have disappeared behind an "awaiting moderation" message after it has already been out there four hours. Nice. But the above link still works (for now).
    - johnchristopher 13y ago
      Yup. I hesitated to paste the post but someone else did it so it's in the wild anyway now.
- cjbprime 13y ago
  Seems to have been moderated away by the blog owners. Classy. Here's the comment under discussion: == Hey. Crazy kids. This probably needs to be that one event where you sort of realize: "Oh. Shit. Other people...like...use this & stuff. We need a damn road map and a release schedule. Stop smoking dabs all day, breh." Now is also a great time to learn how to think about the potential ramifications a production push will have prior to making said production push. And, if your change might impact some or perhaps even all of the other people who use your technology, then some degree of coordination - perhaps an email? - would be nice. It's one of those things that will help make you look professional. I suck at professionalism. You have no idea. But, even I know this much. Because, right now, I sort of feel like I'm asking some very rightfully fearful people to consider entrusting perhaps their actual career into the development of technology they need to succeed and thrive. And, I just started recommending Node.js - with a caveat - that npm basically sucks. I hate having to do that and it needs to stop. So, here we are. Your words continue to be one thing, and your actions continue to be quite another. If it is even possible to break a tool like this, that tool is not enterprise grade. If there is nothing that can be done to successfully insulate a tool from unexpected behavior like this, then that tool scores less in evaluations that consider the risk of using it. npm, at this point, has more going against it in the discussion than going for it right now. Events like this are, in the grand context, very significant and telling. They are also ill-timed. Because big, important decisions are trying to happen right now regarding the use of Node.js. It is literally on the cusp of going mainstream. And, that seems to be generating some pressure that at least one team (npm) doesn't seem to be equipped to handle. So, before you find yourself facing a community that forks instead of trying to work with you, I would like to just make a simple recommendation. In the future, you seriously need to sit and think about the potential ramifications of a production push. And...this is the important part...if those changes are going to have a wide impact on your users - send some sort of email WELL IN ADVANCE. A flippant blog post the day of is not Doing It Right™. Because, and I feel like I might not only be speaking for myself, I'm not going to allow the promise of Node.js to be voided by the lackluster and problematic performance of its weird bolt-on archive service. Someone, perhaps even me (as in: today), will simply replace you with a workable, decentralized solution that enterprise can specialize to purpose and communities can use to grow and thrive. If you have any questions, ask somebody. Anybody. If you're struggling with some concept of enterprise grade operations, what people expect of you and how you can succeed with events like this in the future, I'm positive every capable person here would provide some level of guidance and support. We want you to succeed. Please, try harder or get forked. Not sure how else to say that. Best regards, -Rob
RyanZAG 13y ago
Probably goes down as one of the worst large scale blunders of this type given the sheer amount of people affected. It's actually fairly insane that node.js relies on npm like this - isn't it only a matter of time before one of the core node packages gets compromised and someone gets root access to thousands of servers and dev boxes?
- seiji 13y ago
  Yum (and I seem to recall up2date) have had certificates revoked before (due to compromises on the upstream packaging system) requiring everybody to re-import a good certificate before updates will work again. Now, system updates are a different beast than tying your development process into Magic Hosted Things In Internetland, but certificates on things have been changed before and they will be changed again. Just have to keep aware of how your systems work and what they depend on, which goes against modern "javascript with a double large lack of knowledge" development.
- rhizome 13y ago
  If you're desperate for something, anything, then npm is what you tend to wind up with.
  - mcantelon 13y ago
    Well, it's pretty pleasant to use when there are no issues.
    - rhizome 13y ago
      As is pretty much everything.
      - mcantelon 13y ago
        There are a lot of things I don't find pleasant to use even when there are no issues. Lots of clunky, weirdly designed tech in the world.
- 13y ago
thirsteh 13y ago
Did they previously have client-side validation of the self-signed certificate, and now they changed it to "accept any cert signed by a root CA"? If so, why in the world would they do that? Maybe I'm being hopelessly optimistic by not jumping to the conclusion that they did no validation whatsoever before?
- awj 13y ago
  If running "npm config get ca" on my not-yet-updated copy of npm is any indication, they were using the self signed certificate.
purephase 13y ago
I was bitten by this yesterday and I didn't know WTH was up. This is good to know. I had two or three package installs work flawlessly and then they started failing randomly across servers (all staging, so it wasn't that big of an issue). I think Rob's comment sums up my feelings. A bit more aggressively, but the same gist.
pi-rat 13y ago
Go home NPM, you're drunk! (add all the latest drama, and you're basically just asking to be forked..)
parris 13y ago
Solution 1: Upgrading npm actually does nothing and also complains about a cert error. Solution 2: is a terrible idea for people who have their own private registries or proxy caches. Also a security issue in general... This leaves you with needing to upgrade node, which may or may not be possible due to operating system/platform constraints. Luckily we were using this https://launchpad.net/~chris-lea/+archive/node.js/ https://launchpad.net/~chris-lea/+archive/node.js/ which makes life on ubuntu ok. We set up our own proxy cache, at this point im thinking about increasing the cache time to something like 2 weeks.
jevinskie 13y ago
What broke? Running the npm command? Running any nodejs program? The blog post gives few details about what people were doing when they hit the issue.
- mmorris 13y ago
  You can't npm install <foo> without getting an error message about the self-signed cert. At least until you update npm, which you usually do with npm update -g npm, but that will also fail with the same error. Fun! The blog post doesn't mention deleting the config change after updating, which (I'm not an expert) might be important. I think this is a full fix: http://stackoverflow.com/a/22099006/2708274 http://stackoverflow.com/a/22099006/2708274
  - jevinskie 13y ago
    Did the npm servers previously serve up packages with the self-signed cert?
    - mmorris 13y ago
      Yes.
2mur 13y ago
Basically don't ever deploy from npm. Tarballs
- darkarmani 13y ago
  This is exactly why I don't understand why it is called "package management." What good is NPM if you just need to deploy tarballs anyway? It's good for bloating dependencies and installing 10 of the same library. It completely punts on actually managing packages, by just copying everything -- recursively.
  - ahallock 13y ago
    npm prune
andrewmunsell 13y ago
This is also causing an issue with Dokku-- I'm trying to deploy a new Node app and it fails because of the certificate issue. Anyone know how to update the Buildpack to the latest version? Heroku already implemented a fix.
- agilebyte 13y ago
  Yes I do :) You can see on the following step which image is being used when deploying an app: https://github.com/progrium/dokku/blob/master/dokku#L33 https://github.com/progrium/dokku/blob/master/dokku#L33 So what I did was change that to an image of my own based on the progrium/buildstep one: sudo docker run -i -t progrium/buildstep /bin/bash Now you can make changes to it and save them into your new image (while you are still logged in to progrium/buildstep): sudo docker commit <id> myownimage OK, and inside this image I have referenced my own buildstep instead of the default Heroku one. The relevant txt document is in /build/. And to get to the point I have changed the npm install line to run pointing to the http EU repo. npm install --registry "http://registry.npmjs.eu" --userconfig $build_dir/.npmrc --production 2>&1 | indent That is one way to go about it.
  - andrewmunsell 13y ago
    Great, thanks. I also figured out that I could put the following line into a file called `.env` (I was originally trying to use a file named `ENV`): export BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs This will use Heroku's version of the build pack, which already has a fix for the NPM problem.
IgorPartola 13y ago
Doesn't matter. When you download Node.js itself, you are doing so over HTTP. https://nodejs.org/download/ https://nodejs.org/download/ (note the httpS) does not work. Hope you enjoy being a part of that botnet :) Edit: I used to email project owners about issues like this, but never seem to get any response, so I stopped. The worst one, in terms of losing $$ was Pandora. I keep trying to sign up for the paid service, but the form to put in your credit card details is served over HTTP. I emailed them several times with no response.
- ColdHawaiian 13y ago
  Regarding Pandora, their login form is also served over HTTP instead of HTTPS (http://www.pandora.com/account/sign-in http://www.pandora.com/account/sign-in), which I find to be very disappointing, since it's less secure that way.
- leobelle 13y ago
  Regarding pandora, the form post is over HTTPS: https://www.pandora.com/radio/jsonp/v35 https://www.pandora.com/radio/jsonp/v35 And you can load the form, which is just an HTML input form with some sensitive data (last 4!), over HTTPS. It sucks that HTTPS is not default though.
  - travem 13y ago
    > Regarding pandora, the form post is over HTTPS: Sure, but for the purpose of being explicit to other readers that is still not secure. If the form collecting the sensitive data is served using HTTP only then a malicious actor could alter the page (by injecting some javascript for example) to steal the data even if the actual form post is secured. Serving the form page as https at least ensures that the page collecting the data for submission was unmodified in transit.
    - nevi-me 13y ago
      Hasn't moxie discussed this a few years ago, that it's unsecure to send sensitive data from an HTTP page to an HTTPS one?
janantala 13y ago
Broken builds, broken builds everywhere...
- tracker1 13y ago
  Not here... cache proxy internally.
yogo 13y ago
The inmates are running the asylum :) But it does provide a case study for the rest of us on what or what not to do.
iLoch 13y ago
FYI I was able to bypass this last night by temporarily switching to the EU NPM mirror. npm --registry http://registry.npmjs.eu/ install blah
deleted 13y ago
[deleted]
stevepike 13y ago
Wow, this explains so much. I had to download the coffeescript compiler last night to fix a bug in a library I'm using and hit this error running npm install. I didn't even think to consider that npm itself was broken. I guess it's not always user error!
imslavko 13y ago
Can someone explain what they actually did? What was the technical change that was made? I can't figure out from the blog post whether the change was in the npm server or client.
AdrianRossouw 13y ago
i really don't understand why they would make a server change that would orphan all the existing clients. If anything, roll out a new version and gracefully drop support for other clients.
deelowe 13y ago
Welp that didn't take long. I sure hope someone (who is more skilled than I) forks NPM so we can all move on from this craziness. It's getting pretty sad.
emily37 13y ago
If anyone is wondering what the actual change was: It looks like the npm registry used to have a certificate signed by npm's own CA, and existing npm clients only trust that CA by default, not the normal list of verisign, digicert, etc. (Trusting versign et al would defeat the point of using their own CA.) The signing key for that CA is pretty darn important, and maybe there are entities other than npm, inc who might know it (i.e. nodejitsu). So npm, inc rolled out a new cert that looks to be signed by digicert, but existing clients don't trust Digicert until you explicitly configure them to. I was thrown off by the SELF_SIGNED_CERT_IN_CHAIN error; I expected some error about an untrusted root CA if the problem was that npm clients didn't trust digicert, but apparently SELF_SIGNED_CERT_IN_CHAIN is what OpenSSL returns when the root CA isn't loaded.
- mivok 13y ago
  If the clients trust the npm CA, can't they just sign the digicert CA with that CA and include it in the certificate chain provided by the server? That way the chain would be: npm CA -> digicert CA -> any other intermediates -> server cert Clients that only trust the digicert CA (and other standard CAs) will see that and accept it because they trust the digicert CA, and clients that trust the npm CA will trust the cert also, allowing both old and new clients to work. Once (almost) everyone has upgraded, the npm root CA can be removed from the chain presented by the server. Am I missing something here? Edit: It looks like what I'm missing is that you'd need the private key of the digicert CA to generate the request to sign with the npm CA. I was thinking about how CAs have been migrated in the past (e.g. equifax to geotrust global CA). It looks like it won't work in this case. Edit2: Actually, it appears to work after all. I just tested with the openssl ca command, and you give it -ss_cert instead of -in for the certificate to sign a certificate instead of a request.
  - teacup50 13y ago
    > Am I missing something here? A sense of arrogance that precludes understanding x509 infrastructure before you roll out a world-breaking change.
    - 13y ago
justinfreitag 13y ago
WTH "We are rolling back to the older cert now, but since the registry is distributed by a global CDN this process is slower than we’d like, and we don’t want to break things (further) by rushing the process." http://blog.npmjs.org/post/78165272245/more-help-with-self-signed-cert-in-chain-and-npm http://blog.npmjs.org/post/78165272245/more-help-with-self-s...
automatthew 13y ago
The npm source is crazy. Here's my favorite bugfix (bug being that the prepublish script simply wasn't running): https://github.com/npm/npm/commit/2389525a1df6d20e780cca5887f1b73ff583ce8d https://github.com/npm/npm/commit/2389525a1df6d20e780cca5887...
STRML 13y ago
FYI, npm is rolling back to an older cert that they used a few years back. It's a GlobalSign cert, and the GlobalSign CA has been in npm since Aug 2012. When it propagates their global CDN, old clients will be fixed without breaking new ones and they can focus on a more permanent solution. 1. https://news.ycombinator.com/item?id=7323093 https://news.ycombinator.com/item?id=7323093
deleted 13y ago
[deleted]
novaleaf 13y ago
still broken. following the instructions on windows, I am getting: D:\repos\js-master\phantomjscloud-backend>npm install npm -g --ca="" npm http GET https://registry.npmjs.org/npm https://registry.npmjs.org/npm npm http 200 https://registry.npmjs.org/npm https://registry.npmjs.org/npm npm http GET https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! fetch failed https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm http GET https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! fetch failed https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm http GET https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! fetch failed https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! Error: CERT_UNTRUSTED npm ERR! at SecurePair.<anonymous> (tls.js:1349:32) npm ERR! at SecurePair.EventEmitter.emit (events.js:92:17) npm ERR! at SecurePair.maybeInitFinished (tls.js:962:10) npm ERR! at CleartextStream.read [as _read] (tls.js:463:15) npm ERR! at CleartextStream.Readable.read (_stream_readable.js:320:10) npm ERR! at EncryptedStream.write [as _write] (tls.js:366:25) npm ERR! at doWrite (_stream_writable.js:219:10) npm ERR! at writeOrBuffer (_stream_writable.js:209:5) npm ERR! at EncryptedStream.Writable.write (_stream_writable.js:180:11) npm ERR! at write (_stream_readable.js:573:24) npm ERR! If you need help, you may report this log at: npm ERR! <http://github.com/isaacs/npm/issues> http://github.com/isaacs/npm/issues> npm ERR! or email it to: npm ERR! <npm-@googlegroups.com>
- novaleaf 13y ago
  also tried on my debian 7 server. broken there also.

215 points by rcconf 13y ago

This has also broken all of our own deploys on Jenkins. We had to use solution 2, and then upgrade, because solution 1 also produces the certificate error. It was surprising that npm status didn't have any sort of warning. I had to find out about this via twitter. I think it's extremely irresponsible and I'm glad we've started to move away from node.js and started to use Java.

benajnim 13y ago
That's not a loaded comment.. Changing your programming environment over an SSL certificate? Tell us all about how awesome it is building apps in Java!
- awj 13y ago
  You want to talk loaded comments? You're reducing "I'm glad to move away from a project that breaks functionality for all users with no notice" to a quibble over an SSL certificate change. That npm as a project thought this was the correct way to handle this kind of transition is very unsettling.
  - 1stop 13y ago
    Yeah, because no one ever had any problems with Maven, that thing is like package heaven, it's all rainbows and unicorns!
    - Pacabel 13y ago
      It's really disappointing to see this sort of "reasoning" used so often these days. Just because somebody points out serious flaws or problems with one technology does not automatically mean that he or she thinks that other, unrelated-yet-similar technologies aren't flawed or are somehow perfect.
      - 1stop 13y ago
        > I'm glad we've started to move away from node.js and started to use Java. That is what I was responding to directly. The implication from the statement was that npm flaws (i.e certificate changes that break everything) is a good reason to switch to Java. It isn't at all a good reason to switch to java, as java's equivalent would easily waste more time than even this rather embarrassing certificate problem with npm. > unrelated-yet-similar technologies It is related, and similar, it is a like-for-like comparison between nodejs package management and java package management.
        teacup50 13y ago
        > It isn't at all a good reason to switch to java, as java's equivalent would easily waste more time than even this rather embarrassing certificate problem with npm. Works fine for me. Has a healthy ecosystem of 3rd-party artifact repository implementations.
        deleted 13y ago
        [deleted]
        EdwardDiego 13y ago
        > It isn't at all a good reason to switch to java, as java's equivalent would easily waste more time than even this rather embarrassing certificate problem with npm. As someone who has used it daily for the past 5.5 years - not really. That said, I'd prefer something like Gradle, but I can't fault Maven for just working, goddamnit.
    - awj 13y ago
      How is that comparison at all relevant?
      - 1stop 13y ago
        > I'm glad we've started to move away from node.js and started to use Java. I'm not sure you get how threading works, but this thread originated from the above quote. And comparing npm with Maven given the above statement is relevant, given it is the primary "Package management" system for Java, much like npm.
        awj 13y ago
        I perfectly well understand how threading works. Read the rest of the comment that you cherry picked that quote from, it's about being upset at how npm doesn't seem to be taking it's responsibility seriously. If you had some information about Maven callously performing a user-hostile update, the comparison would be appropriate. As it is you're just relying on "lolz, Java sucks" as a form of argument.
        1stop 13y ago
        No, I'm really not. I don't think java sucks at all, and never said as such. I'm saying Maven isn't better, nor would npm's mistake be a good reason to switch from to Java from Node. npm didn't perform a user-hostile update, they made a mistake with certificate authorities. How many 0-days have forced a java upgrade on people... was that a 'user hostile' move by sun/oracle. I think your line of reasoning is utterly ridiculous, and you are responding to words I didn't say.
- RyanZAG 13y ago
  He never said he was changing because of this issue. (Logically, if he's already been changing over then it can't have been because of an issue that just happened) "Tell us all about how awesome it is building apps in Java!" sounds like a pretty loaded comment to me though.
- nolok 13y ago
  > Changing your programming environment over an SSL certificate? That's not at all what he said.
- twistedpair 13y ago
  In the last 5 years maven (Central) hasn't given me a cert or pgp error. Building 50 times a day in Java is running just fine ;) Of course it could happen in theory to any such similar system such as NuGet if people decided to yank the rug out from under their user base.
tootie 13y ago
Ditto. Team wasted a few hours on this today.

ibash 13y ago

As per Joe Grund: http://blog.npmjs.org/post/78085451721/npms-self-signed-certificate-is-no-more#comment-1264542895 http://blog.npmjs.org/post/78085451721/npms-self-signed-cert... npm config set strict-ssl false npm update npm -g npm config set strict-ssl true

honksillet 13y ago
I was just about to post this link. The change broke bower at on my laptop. The above link fixed it.
awj 13y ago
That's ... still a dangerous way to accomplish the update. You're disabling a key safeguard that ensures you're actually talking to official npm servers, then re-enabling the safeguard after running the code that whoever-you-got-it-from provided. I'm sure the likelihood of experiencing a MITM attack here is low, but the security consideration needs to be as widely advertised as the solution.

atonse 13y ago

I'm surprised that this wasn't one of those "hey in 60 days we'll be making this change and it will break x, y, and z. Be prepared." sort of things.

randunel 13y ago

When package managers misbehave, the whole software bundle suffers. Maybe the Node.JS project should have more (decision) power over its package manager. Node maintainers properly announce api version changes sometimes months in advance, so you don't need to read through the pr/issues on github or mailing lists to stay up to date. I really don't think an advance notice would have hurt. So much for automation.

agilebyte 13y ago
Well, nodejitsu are trademarking NPM so let's see how that plays out.

deleted 13y ago

[deleted]

diminoten 13y ago

There's a comment on the blog post that I thought was very good, it was from Rob, and it ends with: "Please, try harder or get forked. Not sure how else to say that."

johnchristopher 13y ago
Rob's comment was spot on. Here's a link to the full text: http://blog.npmjs.org/post/78085451721/npms-self-signed-certificate-is-no-more#comment-1264657416 http://blog.npmjs.org/post/78085451721/npms-self-signed-cert...
- dullroar 13y ago
  And now that comment seems to have disappeared behind an "awaiting moderation" message after it has already been out there four hours. Nice. But the above link still works (for now).
  - johnchristopher 13y ago
    Yup. I hesitated to paste the post but someone else did it so it's in the wild anyway now.
cjbprime 13y ago
Seems to have been moderated away by the blog owners. Classy. Here's the comment under discussion: == Hey. Crazy kids. This probably needs to be that one event where you sort of realize: "Oh. Shit. Other people...like...use this & stuff. We need a damn road map and a release schedule. Stop smoking dabs all day, breh." Now is also a great time to learn how to think about the potential ramifications a production push will have prior to making said production push. And, if your change might impact some or perhaps even all of the other people who use your technology, then some degree of coordination - perhaps an email? - would be nice. It's one of those things that will help make you look professional. I suck at professionalism. You have no idea. But, even I know this much. Because, right now, I sort of feel like I'm asking some very rightfully fearful people to consider entrusting perhaps their actual career into the development of technology they need to succeed and thrive. And, I just started recommending Node.js - with a caveat - that npm basically sucks. I hate having to do that and it needs to stop. So, here we are. Your words continue to be one thing, and your actions continue to be quite another. If it is even possible to break a tool like this, that tool is not enterprise grade. If there is nothing that can be done to successfully insulate a tool from unexpected behavior like this, then that tool scores less in evaluations that consider the risk of using it. npm, at this point, has more going against it in the discussion than going for it right now. Events like this are, in the grand context, very significant and telling. They are also ill-timed. Because big, important decisions are trying to happen right now regarding the use of Node.js. It is literally on the cusp of going mainstream. And, that seems to be generating some pressure that at least one team (npm) doesn't seem to be equipped to handle. So, before you find yourself facing a community that forks instead of trying to work with you, I would like to just make a simple recommendation. In the future, you seriously need to sit and think about the potential ramifications of a production push. And...this is the important part...if those changes are going to have a wide impact on your users - send some sort of email WELL IN ADVANCE. A flippant blog post the day of is not Doing It Right™. Because, and I feel like I might not only be speaking for myself, I'm not going to allow the promise of Node.js to be voided by the lackluster and problematic performance of its weird bolt-on archive service. Someone, perhaps even me (as in: today), will simply replace you with a workable, decentralized solution that enterprise can specialize to purpose and communities can use to grow and thrive. If you have any questions, ask somebody. Anybody. If you're struggling with some concept of enterprise grade operations, what people expect of you and how you can succeed with events like this in the future, I'm positive every capable person here would provide some level of guidance and support. We want you to succeed. Please, try harder or get forked. Not sure how else to say that. Best regards, -Rob

RyanZAG 13y ago

Probably goes down as one of the worst large scale blunders of this type given the sheer amount of people affected. It's actually fairly insane that node.js relies on npm like this - isn't it only a matter of time before one of the core node packages gets compromised and someone gets root access to thousands of servers and dev boxes?

seiji 13y ago
Yum (and I seem to recall up2date) have had certificates revoked before (due to compromises on the upstream packaging system) requiring everybody to re-import a good certificate before updates will work again. Now, system updates are a different beast than tying your development process into Magic Hosted Things In Internetland, but certificates on things have been changed before and they will be changed again. Just have to keep aware of how your systems work and what they depend on, which goes against modern "javascript with a double large lack of knowledge" development.
rhizome 13y ago
If you're desperate for something, anything, then npm is what you tend to wind up with.
- mcantelon 13y ago
  Well, it's pretty pleasant to use when there are no issues.
  - rhizome 13y ago
    As is pretty much everything.
    - mcantelon 13y ago
      There are a lot of things I don't find pleasant to use even when there are no issues. Lots of clunky, weirdly designed tech in the world.
13y ago

thirsteh 13y ago

Did they previously have client-side validation of the self-signed certificate, and now they changed it to "accept any cert signed by a root CA"? If so, why in the world would they do that? Maybe I'm being hopelessly optimistic by not jumping to the conclusion that they did no validation whatsoever before?

awj 13y ago
If running "npm config get ca" on my not-yet-updated copy of npm is any indication, they were using the self signed certificate.

purephase 13y ago

I was bitten by this yesterday and I didn't know WTH was up. This is good to know. I had two or three package installs work flawlessly and then they started failing randomly across servers (all staging, so it wasn't that big of an issue). I think Rob's comment sums up my feelings. A bit more aggressively, but the same gist.

pi-rat 13y ago

Go home NPM, you're drunk! (add all the latest drama, and you're basically just asking to be forked..)

parris 13y ago

Solution 1: Upgrading npm actually does nothing and also complains about a cert error. Solution 2: is a terrible idea for people who have their own private registries or proxy caches. Also a security issue in general... This leaves you with needing to upgrade node, which may or may not be possible due to operating system/platform constraints. Luckily we were using this https://launchpad.net/~chris-lea/+archive/node.js/ https://launchpad.net/~chris-lea/+archive/node.js/ which makes life on ubuntu ok. We set up our own proxy cache, at this point im thinking about increasing the cache time to something like 2 weeks.

jevinskie 13y ago

What broke? Running the npm command? Running any nodejs program? The blog post gives few details about what people were doing when they hit the issue.

mmorris 13y ago
You can't npm install <foo> without getting an error message about the self-signed cert. At least until you update npm, which you usually do with npm update -g npm, but that will also fail with the same error. Fun! The blog post doesn't mention deleting the config change after updating, which (I'm not an expert) might be important. I think this is a full fix: http://stackoverflow.com/a/22099006/2708274 http://stackoverflow.com/a/22099006/2708274
- jevinskie 13y ago
  Did the npm servers previously serve up packages with the self-signed cert?
  - mmorris 13y ago
    Yes.

2mur 13y ago

Basically don't ever deploy from npm. Tarballs

darkarmani 13y ago
This is exactly why I don't understand why it is called "package management." What good is NPM if you just need to deploy tarballs anyway? It's good for bloating dependencies and installing 10 of the same library. It completely punts on actually managing packages, by just copying everything -- recursively.
- ahallock 13y ago
  npm prune

andrewmunsell 13y ago

This is also causing an issue with Dokku-- I'm trying to deploy a new Node app and it fails because of the certificate issue. Anyone know how to update the Buildpack to the latest version? Heroku already implemented a fix.

agilebyte 13y ago
Yes I do :) You can see on the following step which image is being used when deploying an app: https://github.com/progrium/dokku/blob/master/dokku#L33 https://github.com/progrium/dokku/blob/master/dokku#L33 So what I did was change that to an image of my own based on the progrium/buildstep one: sudo docker run -i -t progrium/buildstep /bin/bash Now you can make changes to it and save them into your new image (while you are still logged in to progrium/buildstep): sudo docker commit <id> myownimage OK, and inside this image I have referenced my own buildstep instead of the default Heroku one. The relevant txt document is in /build/. And to get to the point I have changed the npm install line to run pointing to the http EU repo. npm install --registry "http://registry.npmjs.eu" --userconfig $build_dir/.npmrc --production 2>&1 | indent That is one way to go about it.
- andrewmunsell 13y ago
  Great, thanks. I also figured out that I could put the following line into a file called `.env` (I was originally trying to use a file named `ENV`): export BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs This will use Heroku's version of the build pack, which already has a fix for the NPM problem.

IgorPartola 13y ago

Doesn't matter. When you download Node.js itself, you are doing so over HTTP. https://nodejs.org/download/ https://nodejs.org/download/ (note the httpS) does not work. Hope you enjoy being a part of that botnet :) Edit: I used to email project owners about issues like this, but never seem to get any response, so I stopped. The worst one, in terms of losing $$ was Pandora. I keep trying to sign up for the paid service, but the form to put in your credit card details is served over HTTP. I emailed them several times with no response.

ColdHawaiian 13y ago
Regarding Pandora, their login form is also served over HTTP instead of HTTPS (http://www.pandora.com/account/sign-in http://www.pandora.com/account/sign-in), which I find to be very disappointing, since it's less secure that way.
leobelle 13y ago
Regarding pandora, the form post is over HTTPS: https://www.pandora.com/radio/jsonp/v35 https://www.pandora.com/radio/jsonp/v35 And you can load the form, which is just an HTML input form with some sensitive data (last 4!), over HTTPS. It sucks that HTTPS is not default though.
- travem 13y ago
  > Regarding pandora, the form post is over HTTPS: Sure, but for the purpose of being explicit to other readers that is still not secure. If the form collecting the sensitive data is served using HTTP only then a malicious actor could alter the page (by injecting some javascript for example) to steal the data even if the actual form post is secured. Serving the form page as https at least ensures that the page collecting the data for submission was unmodified in transit.
  - nevi-me 13y ago
    Hasn't moxie discussed this a few years ago, that it's unsecure to send sensitive data from an HTTP page to an HTTPS one?

janantala 13y ago

Broken builds, broken builds everywhere...

tracker1 13y ago
Not here... cache proxy internally.

yogo 13y ago

The inmates are running the asylum :) But it does provide a case study for the rest of us on what or what not to do.

iLoch 13y ago

FYI I was able to bypass this last night by temporarily switching to the EU NPM mirror. npm --registry http://registry.npmjs.eu/ install blah

deleted 13y ago

[deleted]

stevepike 13y ago

Wow, this explains so much. I had to download the coffeescript compiler last night to fix a bug in a library I'm using and hit this error running npm install. I didn't even think to consider that npm itself was broken. I guess it's not always user error!

imslavko 13y ago

Can someone explain what they actually did? What was the technical change that was made? I can't figure out from the blog post whether the change was in the npm server or client.

AdrianRossouw 13y ago

i really don't understand why they would make a server change that would orphan all the existing clients. If anything, roll out a new version and gracefully drop support for other clients.

deelowe 13y ago

Welp that didn't take long. I sure hope someone (who is more skilled than I) forks NPM so we can all move on from this craziness. It's getting pretty sad.

emily37 13y ago

If anyone is wondering what the actual change was: It looks like the npm registry used to have a certificate signed by npm's own CA, and existing npm clients only trust that CA by default, not the normal list of verisign, digicert, etc. (Trusting versign et al would defeat the point of using their own CA.) The signing key for that CA is pretty darn important, and maybe there are entities other than npm, inc who might know it (i.e. nodejitsu). So npm, inc rolled out a new cert that looks to be signed by digicert, but existing clients don't trust Digicert until you explicitly configure them to. I was thrown off by the SELF_SIGNED_CERT_IN_CHAIN error; I expected some error about an untrusted root CA if the problem was that npm clients didn't trust digicert, but apparently SELF_SIGNED_CERT_IN_CHAIN is what OpenSSL returns when the root CA isn't loaded.

mivok 13y ago
If the clients trust the npm CA, can't they just sign the digicert CA with that CA and include it in the certificate chain provided by the server? That way the chain would be: npm CA -> digicert CA -> any other intermediates -> server cert Clients that only trust the digicert CA (and other standard CAs) will see that and accept it because they trust the digicert CA, and clients that trust the npm CA will trust the cert also, allowing both old and new clients to work. Once (almost) everyone has upgraded, the npm root CA can be removed from the chain presented by the server. Am I missing something here? Edit: It looks like what I'm missing is that you'd need the private key of the digicert CA to generate the request to sign with the npm CA. I was thinking about how CAs have been migrated in the past (e.g. equifax to geotrust global CA). It looks like it won't work in this case. Edit2: Actually, it appears to work after all. I just tested with the openssl ca command, and you give it -ss_cert instead of -in for the certificate to sign a certificate instead of a request.
- teacup50 13y ago
  > Am I missing something here? A sense of arrogance that precludes understanding x509 infrastructure before you roll out a world-breaking change.
  - 13y ago

justinfreitag 13y ago

WTH "We are rolling back to the older cert now, but since the registry is distributed by a global CDN this process is slower than we’d like, and we don’t want to break things (further) by rushing the process." http://blog.npmjs.org/post/78165272245/more-help-with-self-signed-cert-in-chain-and-npm http://blog.npmjs.org/post/78165272245/more-help-with-self-s...

automatthew 13y ago

The npm source is crazy. Here's my favorite bugfix (bug being that the prepublish script simply wasn't running): https://github.com/npm/npm/commit/2389525a1df6d20e780cca5887f1b73ff583ce8d https://github.com/npm/npm/commit/2389525a1df6d20e780cca5887...

STRML 13y ago

FYI, npm is rolling back to an older cert that they used a few years back. It's a GlobalSign cert, and the GlobalSign CA has been in npm since Aug 2012. When it propagates their global CDN, old clients will be fixed without breaking new ones and they can focus on a more permanent solution. 1. https://news.ycombinator.com/item?id=7323093 https://news.ycombinator.com/item?id=7323093

deleted 13y ago

[deleted]

novaleaf 13y ago

still broken. following the instructions on windows, I am getting: D:\repos\js-master\phantomjscloud-backend>npm install npm -g --ca="" npm http GET https://registry.npmjs.org/npm https://registry.npmjs.org/npm npm http 200 https://registry.npmjs.org/npm https://registry.npmjs.org/npm npm http GET https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! fetch failed https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm http GET https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! fetch failed https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm http GET https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! fetch failed https://registry.npmjs.org/npm/-/npm-1.4.4.tgz https://registry.npmjs.org/npm/-/npm-1.4.4.tgz npm ERR! Error: CERT_UNTRUSTED npm ERR! at SecurePair.<anonymous> (tls.js:1349:32) npm ERR! at SecurePair.EventEmitter.emit (events.js:92:17) npm ERR! at SecurePair.maybeInitFinished (tls.js:962:10) npm ERR! at CleartextStream.read [as _read] (tls.js:463:15) npm ERR! at CleartextStream.Readable.read (_stream_readable.js:320:10) npm ERR! at EncryptedStream.write [as _write] (tls.js:366:25) npm ERR! at doWrite (_stream_writable.js:219:10) npm ERR! at writeOrBuffer (_stream_writable.js:209:5) npm ERR! at EncryptedStream.Writable.write (_stream_writable.js:180:11) npm ERR! at write (_stream_readable.js:573:24) npm ERR! If you need help, you may report this log at: npm ERR! <http://github.com/isaacs/npm/issues> http://github.com/isaacs/npm/issues> npm ERR! or email it to: npm ERR! <npm-@googlegroups.com>

novaleaf 13y ago
also tried on my debian 7 server. broken there also.