8 ms·

Godaddy has issued at least 8850 SSL certificates without validating anything

327 points by 0x0 10y ago

jessaustin 10y ago
According to TFA, the certs have already been revoked.
- johansch 10y ago
  Relevant: https://www.maikel.pro/blog/current-state-certificate-revocation-crls-ocsp/ https://www.maikel.pro/blog/current-state-certificate-revoca...
  - EvilTerran 10y ago
    Great article! Just the sort of thing I come to HN for. I note it's a year old, though - has much changed since then? OCSP stapling sounds like a Very Good Idea, I hope it's been catching on.
    - pfg 10y ago
      Not all that much has changed. SSL Pulse[1] shows that use of OCSP Stapling went up about 3% in the past 12 months, currently putting it at 25% of observed sites. OCSP Must-Staple is now available in Firefox and can be added as an extension to certificates issued by Let's Encrypt. No other browsers have plans for Must-Staple AFAIK. Otherwise, OCSP is still soft-fail for everything but EV in most scenarios. [1]: https://www.trustworthyinternet.org/ssl-pulse/ https://www.trustworthyinternet.org/ssl-pulse/
      - EvilTerran 10y ago
        +3%? Well, that's less inspiring news than I'd hoped. Still, it's something, I suppose. Anyway, that's another very interesting link you've got there - I knew about SSL Labs, but hadn't come across that report before. Looks like something worth keeping an eye on. Cheers!
    - hannob 10y ago
      The problem is that the major web server apache has basically a broken ocsp stapling implementation. It does a bunch of unexpected things and you'll likely not want to enforce stapling with it.
aaronpk 10y ago
This sounds like responsible handling, disclosure, and remedying of the problem.
Scaevolus 10y ago
tl;dr: they requested a URL and wanted it to echo a token passed in the query string. They accepted 404 pages that echoed the token as valid, too! "In case anyone is wondering why this is problematic, during the Ballot 169 review process, Peter Bowen ran a check against the top 10,000 Alexa domains and noted that more than 400 sites returned a HTTP 200 response for a request to http://www.$DOMAIN/.well-known/pki-validation/4c079484040e32529577b6a5aade31c5af6fe0c7 http://www.$DOMAIN/.well-known/pki-validation/4c079484040e32... [1]. A number of those included the URL in the response body, which would presumably be good enough for GoDaddy's domain validation process if they indeed only check for a HTTP 200 response. [1]: https://cabforum.org/pipermail/public/2016-April/007506.html https://cabforum.org/pipermail/public/2016-April/007506.html "
- akjainaj 10y ago
  They needed the page to return just the token or something with the token in it? Because then I could validate this: http://google.com/this_is_my_token http://google.com/this_is_my_token
  - duskwuff 10y ago
    Just something with the token in it. So, yes. You probably wouldn't have been able to validate google.com, as GoDaddy would have (hopefully) flagged any domains containing major trademarks like "google" for manual review, but that's hardly a strong protection.
    - akjainaj 10y ago
      Everybody who's used the Internet for more than two days knows that many 404 pages have the URL inside the html (Apache default 404 comes to my mind), so I suppose this was made by someone who simply didn't give a fuck. Which is kinda big deal if you're generating SSL certs...
      - geofft 10y ago
        1. It sounds like this was originally implemented correctly, and a code change caused the check for status 200 to stop working. (See the first message, sentence starting "A configuration change to the library") 2. There are a number of "404" pages that are sent with HTTP status 200 and also echo the URL inside the HTML. See https://cabforum.org/pipermail/public/2016-April/007506.html https://cabforum.org/pipermail/public/2016-April/007506.html , which Patrick Figel linked to in the thread.
sebcat 10y ago
- request http://example.com/<path http://example.com/<path or query containing random token> - if random token is echoed in the response, and the HTTP response code is 200, they consider that the applicant has control over the requested FQDN Do I understand their validation method correctly? If so, I wouldn't consider it very secure.
- kodablah 10y ago
  I believe this is essentially what ACME does when using the domain challenge, granted it likely requires a certain set of content to ensure your misconfigured webserver doesn't return 200 for all paths.
  - ploxiln 10y ago
    ACME does not include the expected response in the path or query. So it can't be accidentally included in the response. EDIT: quote by Nick Lamb in the discussion linked: > ACME http-01 won't get tripped here because the checked content of the URL is very much not the random string (it's a JWS signature over a data structure containing that random string, thereby proving it was made by whoever the ACME server is talking to). But yes, doing something that _looks_ superficially like the ACME style of validation without such subtlety will trip you up.
    - deleted 10y ago
      [deleted]
  - eridius 10y ago
    I haven't looked at ACME, but any properly-designed system should require the entire response to match some expected value instead of just searching for the value as a substring.
  - andrewstuart2 10y ago
    As ploxiln mentioned, there's a lot more to ACME. ACME uses challenge-response validation, which is significantly more complicated than an echo. You've already sent a CSR to LetsEncrypt that contains your server's generated public key. LetsEncrypt then issues a challenge that your server must sign using the associated private key that it generated, and that only it has access to. Then LetsEncrypt validates the signature using the public key and checks that a) the signed data matches the nonce they sent as a challenge and b) the signature is valid given the known public key. This sufficiently guarantees the fact that you control both the domain name and the key associated with your submitted CSR. At that point, they sign and issue your certificate and send it back to you.
pfg 10y ago
Interesting tidbit: The CA/B Forum passed a change to the Baseline Requirements attempting to standardize the methods of domain ownership validation back in August of last year[1]. Prior to that, it was essentially up to the CAs to come up with secure methods. The methods described in that change contained mitigations against this vulnerability. The change never went into effect (practically speaking - it's actually a bit more complex) because a number of CAs in the Forum filed patent exclusion notices, and wouldn't you know it: GoDaddy was one of them. Hope it was worth it. [1]: https://cabforum.org/2016/08/05/ballot-169-revised-validation-requirements/ https://cabforum.org/2016/08/05/ballot-169-revised-validatio... [2]: https://cabforum.org/wp-content/uploads/GoDaddy-Ballot-169-Exclusion-Notice.pdf https://cabforum.org/wp-content/uploads/GoDaddy-Ballot-169-E...
charonn0 10y ago
> Prior to the bug, the library used to query the website and check for the code was configured to return a failure if the HTTP status code was not 200 (success). A configuration change to the library caused it to return results even when the HTTP status code was not 200. Since many web servers are configured to include the URL of the request in the body of a 404 (not found) response, and the URL also contained the random code, any web server configured this way caused domain control verification to complete successfully. I'd bet that the library in question was libcurl, and they forgot to set CURLOPT_FAILONERROR[1]. [1]: https://curl.haxx.se/libcurl/c/CURLOPT_FAILONERROR.html https://curl.haxx.se/libcurl/c/CURLOPT_FAILONERROR.html
deleted 10y ago
[deleted]
yeukhon 10y ago
I own a SSL certificate when I bought a domain from NameCheap. The SSL cert is provided by Comodo. I recently lost my box so I also lost my cert and had to revoke the existing one and ask to re-generate. I would need to wait ~3-6 hours before a new cert is available (ugh), so I decided to go to Let's Encrypt. While the initial setup for starter is quite confusing, I managed to create one regardless and I haven't looked back. I can't help but to think when someone was designing this challenge scheme he/she must have thought of this potential risk, but probably shrug it because "most people won't be able to come up with this method."
ydoxy 10y ago
you are right. I fully agree with you Seo Expert http://www.ydoxy.com http://www.ydoxy.com
simonjgreen 10y ago
Several of my customers were hit by this. What concerns me possibly even more is that GoDaddy, having revoked the certificates, then managed to "un-revoke" them on request with a grace period. This is unsettling, it's not how the CRL system is supposed to work!
- an_account_name 10y ago
  That's... troubling. You should consider mentioning it on the mozilla.dev.security.policy thread: https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/Htujoyq-pO8 https://groups.google.com/forum/#!topic/mozilla.dev.security... EDIT: GoDaddy themselves say they will never do this: https://www.godaddy.com/help/revoke-a-certificate-4747 https://www.godaddy.com/help/revoke-a-certificate-4747 "The process cannot be reversed."
  - simonjgreen 10y ago
    https://medium.com/@SimonJGreen/godaddy-revoked-and-then-un-revoked-a-certificate-without-request-11f86074bbf8#.wyx5czj27 https://medium.com/@SimonJGreen/godaddy-revoked-and-then-un-...
    - tialaramex 10y ago
      The first part of the story definitely checks out https://crt.sh/?id=29236482 https://crt.sh/?id=29236482 This certificate absolutely was revoked by GoDaddy However, that certificate is _still_ revoked right now. A _new_ certificate for the same names was issued on the 12th of January, presumably once the re-validation was completed. This isn't in violation of any policies. Sites on that new certificate such as https://royalduchy.co.uk/ https://royalduchy.co.uk/ do indeed work fine. Can you update the medium story to reflect this? I mean, not your feelings about GoDaddy, say whatever you feel, but the facts aren't as portrayed in that story so far as I am able to see.
Eun 10y ago
Finder here, this is the history: 12.12.2016: First contact with MS 27.12.2016: Answer, saying its not a bug. (Notice the promise that you get an answer in 24 hours) 02.01.2017: Explaining the issue again in more detail 03.01.2017: Opening the ticket, saying I will get more information if something is available. 12.01.2017: No answer from MS, and seeing randomly this post on HN I hope I will get some details by MS soon so I can keep you up to date guys.

327 points by 0x0 10y ago

jessaustin 10y ago

According to TFA, the certs have already been revoked.

johansch 10y ago
Relevant: https://www.maikel.pro/blog/current-state-certificate-revocation-crls-ocsp/ https://www.maikel.pro/blog/current-state-certificate-revoca...
- EvilTerran 10y ago
  Great article! Just the sort of thing I come to HN for. I note it's a year old, though - has much changed since then? OCSP stapling sounds like a Very Good Idea, I hope it's been catching on.
  - pfg 10y ago
    Not all that much has changed. SSL Pulse[1] shows that use of OCSP Stapling went up about 3% in the past 12 months, currently putting it at 25% of observed sites. OCSP Must-Staple is now available in Firefox and can be added as an extension to certificates issued by Let's Encrypt. No other browsers have plans for Must-Staple AFAIK. Otherwise, OCSP is still soft-fail for everything but EV in most scenarios. [1]: https://www.trustworthyinternet.org/ssl-pulse/ https://www.trustworthyinternet.org/ssl-pulse/
    - EvilTerran 10y ago
      +3%? Well, that's less inspiring news than I'd hoped. Still, it's something, I suppose. Anyway, that's another very interesting link you've got there - I knew about SSL Labs, but hadn't come across that report before. Looks like something worth keeping an eye on. Cheers!
  - hannob 10y ago
    The problem is that the major web server apache has basically a broken ocsp stapling implementation. It does a bunch of unexpected things and you'll likely not want to enforce stapling with it.

aaronpk 10y ago

This sounds like responsible handling, disclosure, and remedying of the problem.

Scaevolus 10y ago

tl;dr: they requested a URL and wanted it to echo a token passed in the query string. They accepted 404 pages that echoed the token as valid, too! "In case anyone is wondering why this is problematic, during the Ballot 169 review process, Peter Bowen ran a check against the top 10,000 Alexa domains and noted that more than 400 sites returned a HTTP 200 response for a request to http://www.$DOMAIN/.well-known/pki-validation/4c079484040e32529577b6a5aade31c5af6fe0c7 http://www.$DOMAIN/.well-known/pki-validation/4c079484040e32... [1]. A number of those included the URL in the response body, which would presumably be good enough for GoDaddy's domain validation process if they indeed only check for a HTTP 200 response. [1]: https://cabforum.org/pipermail/public/2016-April/007506.html https://cabforum.org/pipermail/public/2016-April/007506.html "

akjainaj 10y ago
They needed the page to return just the token or something with the token in it? Because then I could validate this: http://google.com/this_is_my_token http://google.com/this_is_my_token
- duskwuff 10y ago
  Just something with the token in it. So, yes. You probably wouldn't have been able to validate google.com, as GoDaddy would have (hopefully) flagged any domains containing major trademarks like "google" for manual review, but that's hardly a strong protection.
  - akjainaj 10y ago
    Everybody who's used the Internet for more than two days knows that many 404 pages have the URL inside the html (Apache default 404 comes to my mind), so I suppose this was made by someone who simply didn't give a fuck. Which is kinda big deal if you're generating SSL certs...
    - geofft 10y ago
      1. It sounds like this was originally implemented correctly, and a code change caused the check for status 200 to stop working. (See the first message, sentence starting "A configuration change to the library") 2. There are a number of "404" pages that are sent with HTTP status 200 and also echo the URL inside the HTML. See https://cabforum.org/pipermail/public/2016-April/007506.html https://cabforum.org/pipermail/public/2016-April/007506.html , which Patrick Figel linked to in the thread.

sebcat 10y ago

- request http://example.com/<path http://example.com/<path or query containing random token> - if random token is echoed in the response, and the HTTP response code is 200, they consider that the applicant has control over the requested FQDN Do I understand their validation method correctly? If so, I wouldn't consider it very secure.

kodablah 10y ago
I believe this is essentially what ACME does when using the domain challenge, granted it likely requires a certain set of content to ensure your misconfigured webserver doesn't return 200 for all paths.
- ploxiln 10y ago
  ACME does not include the expected response in the path or query. So it can't be accidentally included in the response. EDIT: quote by Nick Lamb in the discussion linked: > ACME http-01 won't get tripped here because the checked content of the URL is very much not the random string (it's a JWS signature over a data structure containing that random string, thereby proving it was made by whoever the ACME server is talking to). But yes, doing something that _looks_ superficially like the ACME style of validation without such subtlety will trip you up.
  - deleted 10y ago
    [deleted]
- eridius 10y ago
  I haven't looked at ACME, but any properly-designed system should require the entire response to match some expected value instead of just searching for the value as a substring.
- andrewstuart2 10y ago
  As ploxiln mentioned, there's a lot more to ACME. ACME uses challenge-response validation, which is significantly more complicated than an echo. You've already sent a CSR to LetsEncrypt that contains your server's generated public key. LetsEncrypt then issues a challenge that your server must sign using the associated private key that it generated, and that only it has access to. Then LetsEncrypt validates the signature using the public key and checks that a) the signed data matches the nonce they sent as a challenge and b) the signature is valid given the known public key. This sufficiently guarantees the fact that you control both the domain name and the key associated with your submitted CSR. At that point, they sign and issue your certificate and send it back to you.

pfg 10y ago

Interesting tidbit: The CA/B Forum passed a change to the Baseline Requirements attempting to standardize the methods of domain ownership validation back in August of last year[1]. Prior to that, it was essentially up to the CAs to come up with secure methods. The methods described in that change contained mitigations against this vulnerability. The change never went into effect (practically speaking - it's actually a bit more complex) because a number of CAs in the Forum filed patent exclusion notices, and wouldn't you know it: GoDaddy was one of them. Hope it was worth it. [1]: https://cabforum.org/2016/08/05/ballot-169-revised-validation-requirements/ https://cabforum.org/2016/08/05/ballot-169-revised-validatio... [2]: https://cabforum.org/wp-content/uploads/GoDaddy-Ballot-169-Exclusion-Notice.pdf https://cabforum.org/wp-content/uploads/GoDaddy-Ballot-169-E...

charonn0 10y ago

> Prior to the bug, the library used to query the website and check for the code was configured to return a failure if the HTTP status code was not 200 (success). A configuration change to the library caused it to return results even when the HTTP status code was not 200. Since many web servers are configured to include the URL of the request in the body of a 404 (not found) response, and the URL also contained the random code, any web server configured this way caused domain control verification to complete successfully. I'd bet that the library in question was libcurl, and they forgot to set CURLOPT_FAILONERROR[1]. [1]: https://curl.haxx.se/libcurl/c/CURLOPT_FAILONERROR.html https://curl.haxx.se/libcurl/c/CURLOPT_FAILONERROR.html

deleted 10y ago

[deleted]

yeukhon 10y ago

I own a SSL certificate when I bought a domain from NameCheap. The SSL cert is provided by Comodo. I recently lost my box so I also lost my cert and had to revoke the existing one and ask to re-generate. I would need to wait ~3-6 hours before a new cert is available (ugh), so I decided to go to Let's Encrypt. While the initial setup for starter is quite confusing, I managed to create one regardless and I haven't looked back. I can't help but to think when someone was designing this challenge scheme he/she must have thought of this potential risk, but probably shrug it because "most people won't be able to come up with this method."

ydoxy 10y ago

you are right. I fully agree with you Seo Expert http://www.ydoxy.com http://www.ydoxy.com

simonjgreen 10y ago

Several of my customers were hit by this. What concerns me possibly even more is that GoDaddy, having revoked the certificates, then managed to "un-revoke" them on request with a grace period. This is unsettling, it's not how the CRL system is supposed to work!

an_account_name 10y ago
That's... troubling. You should consider mentioning it on the mozilla.dev.security.policy thread: https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/Htujoyq-pO8 https://groups.google.com/forum/#!topic/mozilla.dev.security... EDIT: GoDaddy themselves say they will never do this: https://www.godaddy.com/help/revoke-a-certificate-4747 https://www.godaddy.com/help/revoke-a-certificate-4747 "The process cannot be reversed."
- simonjgreen 10y ago
  https://medium.com/@SimonJGreen/godaddy-revoked-and-then-un-revoked-a-certificate-without-request-11f86074bbf8#.wyx5czj27 https://medium.com/@SimonJGreen/godaddy-revoked-and-then-un-...
  - tialaramex 10y ago
    The first part of the story definitely checks out https://crt.sh/?id=29236482 https://crt.sh/?id=29236482 This certificate absolutely was revoked by GoDaddy However, that certificate is _still_ revoked right now. A _new_ certificate for the same names was issued on the 12th of January, presumably once the re-validation was completed. This isn't in violation of any policies. Sites on that new certificate such as https://royalduchy.co.uk/ https://royalduchy.co.uk/ do indeed work fine. Can you update the medium story to reflect this? I mean, not your feelings about GoDaddy, say whatever you feel, but the facts aren't as portrayed in that story so far as I am able to see.

Eun 10y ago

Finder here, this is the history: 12.12.2016: First contact with MS 27.12.2016: Answer, saying its not a bug. (Notice the promise that you get an answer in 24 hours) 02.01.2017: Explaining the issue again in more detail 03.01.2017: Opening the ticket, saying I will get more information if something is available. 12.01.2017: No answer from MS, and seeing randomly this post on HN I hope I will get some details by MS soon so I can keep you up to date guys.