13 ms·

Distrusting New WoSign and StartCom Certificates

132 points by buovjaga 10y ago

markild 10y ago
WoSign's response[1] might also be of interest in this context. They seem to take, at least partially, ownership of the issues. Am I also correct in that this only affects intermediate certificates? I see they say that they will have a workaround in place in about a months time. [1]: https://www.wosign.com/English/News/announcement_about_Mozilla_Action_20161024.htm https://www.wosign.com/English/News/announcement_about_Mozil...
- jlgaddis 10y ago
  > Am I also correct in that this only affects intermediate certificates? No. As the article states, it will affect any certificates that chain up to the specified root certificates (including and intermediate and end-entity certificates). > I see they say that they will have a workaround in place ... From previous statements, I believe WoSign's plan is to resell another CA's certificates during the period that they don't have a root of their own in the trust store.
  - pilif 10y ago
    > From previous statements, I believe WoSign's plan is to resell another CA's certificates during the period that they don't have a root of their own in the trust store. or find/buy another currently trusted CA that cross-signs their new root. I'm sure they can find somebody. edit: yes. that's what they are going to do according to https://www.wosign.com/English/News/announcement_about_Mozilla_Action_20161024.htm https://www.wosign.com/English/News/announcement_about_Mozil...: > There will be new SSL certificates issued by a new WoSign intermediate CA which is signed by the one of global trusted root CA, it supports all the browsers (including Firefox). This will be done within one months. I wonder who's going to be stupid/reckless enough to sign that intermediate.
    - tankenmate 10y ago
      Indeed, any company that signs any of their intermediates would want to have an ironclad indemnification, penalty clauses and a fully paid up escrow (hosted in a neutral country like Switzerland, Ireland or the like) in any contract. But I suspect that another Chinese company or Chinese owned company will be selected to be the signor.
    - pfg 10y ago
      It remains to be seen whether they will actually hold the private key for that intermediate certificate and issue end-entity certificates from it, or if this is just some sort of reselling deal where a different, trusted CA holds the key, performs domain validation, etc (which is a fairly common practice). I have my doubts about whether Mozilla will accept them continuing to operate an actual CA with a new cross-signed certificate prior to them completing the inclusion process. CAs need to disclose these intermediate certificates, and I expect it would end up being revoked, with possible sanctions for whoever cross-signs them.
    - bandrami 10y ago
      I wonder who's going to be stupid/reckless enough to sign that intermediate. You have a much higher opinion of the probity (and competence) of CAs than is probably warranted.
- TravelTechGuy 10y ago
  I've noticed 2 things in their response: 1. They take no direct responsibility, nor try to explain, or excuse the deception claims. To WoSign, the whole thing is "an incident", not a premeditated deception. 2. They still do not acknowledge their ownership of StartCom, or explain why that was kept in the dark. In fact they keep talking about "4 WoSign roots" despite the 6 roots mentioned by Mozilla.
jo909 10y ago
I am kind of thankful for this. It gave us immediate time budget to switch our certificate process over to Letsencrypt. I'm also glad it does not affect existing certs, since we need some weeks to ramp up the LE certs because of the 20 new certs per week limit (we can continue to only bundle subdomains per cert that are actually used on the same host/loadbalancer).
- emilburzo 10y ago
  Do you have multiple hosts serving the same domain by any chance? I'm wondering if there's been any progress in finding an easy solution for this use-case.
  - pilif 10y ago
    Many if not all of the acme clients out there store the key and certificate as a file. Just copy that to the various machines terminating the SSL connection. In our case, there's a script on the frontend machine running HAProxy that's fetching the certificates and putting them on a shared file system so the backup machine also has access to them. The traffic between frontend host and application servers is unencrypted in our case as it's all one rack under our control. If it wasn't, we'd be using a self-signed certificate for that connection.
0x0 10y ago
Interestingly, over at the Google Certificate Transparency group, it looks like both StartCom's and WoSign's CT log servers were recently included into Chrome. https://groups.google.com/a/chromium.org/forum/#!topic/ct-policy/4BNen-sogf8 https://groups.google.com/a/chromium.org/forum/#!topic/ct-po...
- iancarroll 10y ago
  Luckily CT logs are designed to be mostly trustless.
  - 0x0 10y ago
    Yet the Izenpe CT log managed to be untrustworthy enough to be de-listed! https://groups.google.com/a/chromium.org/forum/#!topic/ct-policy/g0WWgUodpOk https://groups.google.com/a/chromium.org/forum/#!topic/ct-po...
    - iancarroll 10y ago
      Indeed. For those curious, CT is designed to "discover" SCTs that were not publicly shown by the log, so that the log can operate without a lot of trust, though this did not happen here (someone just found them operating two logs...) I guess I'm not the best person to criticize other logs, though. :-)
- dorianm 10y ago
  "Please note that while both Symantec and CNNIC's logs are presently enabled in the current Chrome stable release (Chrome 52), both WoSign and StartCom's logs will not be recognized until Chrome 54, which is presently the Dev channel release."
  - randomfool 10y ago
    https://googlechromereleases.blogspot.com/2016/10/stable-channel-update-for-desktop_20.html https://googlechromereleases.blogspot.com/2016/10/stable-cha... Chrome 54 is now in stable.
jpablo 10y ago
I don't like this at all. The only value left in WoSign root certificates is in issuing backdated certificates that wouldn't be widely distributed. So basically the only way to extract any money out of their current root keys is to sell rogue certificates for targeted attacks.
- BillinghamJ 10y ago
  Mozilla has explicitly stated that if they find any evidence of this happening, both WoSign and StartCom will be completely revoked - both for old and new certificates. > If additional back-dating is discovered (by any means) to circumvent this control, then Mozilla will immediately and permanently revoke trust in the affected roots.
  - jamiesonbecker 10y ago
    > if they find any evidence of this happening There's already evidence of this happening. It happened. Why does it have to happen again. Just revoke em.
    - inimino 10y ago
      The point of doing it this way is that it puts the CA out of business but doesn't break the Web, which makes it the only really practical way for browsers to exert leverage over too big to fail CA's.
      - lathiat 10y ago
        more accurately, this way all the existing issued certificates are not invalidated. the idea is not to punish the customers of wosign, startcom who already paid for a certificate and have it in use. if they removed it entirely, those would all break which would be inconvenient to the otherwise innocent customers.
        jamiesonbecker 10y ago
lexman0 10y ago
"Although Mozilla’s sanctions are too severe..." These guys must be joking. Trust has been lost, the roots should be permanently revoked. If anything, I think Mozilla's actions are not severe enough. How likely is it that Mozilla doesn't know the full story? There may be additional violations that have been missed.
- jcranmer 10y ago
  Mozilla has revoked the CA certificate moving forward. Already-issued certificates are unaffected, to allow current customers time to migrate to a more reputable CA. If they were to immediately revoke the current roots, then thousands of sites would suddenly report certificate errors--which would train users to click through them, helping nobody. The axe Mozilla is holding over WoSign is the threat of immediate full revocation if WoSign is caught doing backdating again. Given that WoSign has been coerced into cooperating with publishing all CSRs via Certificate Transparency, and that there is likely to be a much larger group of people watching carefully for violations, I don't expect it to take very long for future backdated certificates to be caught if WoSign does try it.
bandrami 10y ago
Maybe I'm reading this wrong, but doesn't this mean that the actual, found-in-the-wild backdated certs will still be trusted? At any rate, this gets to the crux of PKI's problem. This was a backdating of certificates because of a change in policy about the cryptographic strength of hashes. But the weak point in PKI isn't the cryptography, it's the agents.
- pfg 10y ago
  IIRC they were revoked in the course of the investigation (somewhere in the last two months). That being said, the risky bit is that they actually signed those SHA-1 certificates in the first place - a certificate that could be specially crafted to "collide" with one for a site an attacker wants to impersonate. It's not really all that important whether they're revoked (though there's nothing wrong with revoking them just in case). As for CAs being the weak point, luckily we're going to be living in a mandatory Certificate Transparency world in the not-so-far future.
  - bandrami 10y ago
    IDK, I'm worried this is an example of what that will look like: since the CAs are the actual stakeholders, the default question is going to be "what can we do to make this CA trustworthy again?" rather than "how can we make sure this CA never has another cert trusted again?"
- jcranmer 10y ago
  The backdated certificates that have already been found are explicitly being added to OneCRL, which is to say that they are explicitly distrusted independent of what the CRL/OSCP responder of WoSign and StartCom say.
  - bandrami 10y ago
    So some portion of Firefox users will be protected from them. Better than nothing I guess.
jamiesonbecker 10y ago
It'd be nice if we could distrust the whole CA system and start from scratch. (of course, to everyone who is saying to themselves that this is absurd and unrealistic.. yes, of course you're right.. for the moment.) It seems pretty clear that the future lies in the blockchain.
- dpark 10y ago
  How does the blockchain solve the problems with CAs?
  - pfg 10y ago
    Haven't you heard, you can solve any problem by throwing a blockchain at it! ;-) On a more serious note, Certificate Transparency, which will become mandatory in Chrome in about a year, uses technology that is strongly related to blockchains, so we're really not too far from a blockchain-like solution for the Web PKI.
    - Torgo 10y ago
      A global, permanent, public, append-only log for 11 cents a record is pretty useful.
    - schoen 10y ago
      I agree that CT uses a technology related to blockchains, but as I noted above, Ben Laurie has been pretty vocal in saying that the Bitcoin-style PoW blockchain isn't a technology that he favors, and I think he thinks CT is importantly different in some respects (for example in accepting more centralization in the operation of logs: an uninvolved anonymous party can't show up and "mine" a CT log event). http://www.links.org/files/decentralised-currencies.pdf http://www.links.org/files/decentralised-currencies.pdf
  - throwaway7767 10y ago
    Not the OP, but I imagine he has something like namecoin in mind. Domains are tied to cryptographic keys which are also used for encryption+authentication, and the blockchain stores the mapping of names to keys. It's not ready for primetime, and it has its own issues. I don't really see it gaining much traction. But it does present an alternative approach to the problem.
JumpCrisscross 10y ago
> No longer accept audits carried out by Ernst & Young Hong Kong This is...a big deal? If Mozilla believes E&Y HK was negligent or complicit in WoSign's lies, then that newfound scepticism should extend to the office's financial audits.
- throwaway7767 10y ago
  Seems weird to tie it just to the Hong Kong office though. These guys are trading on the Ernst & Young name to get the positive reputation associated with it, why should the negative publicity not flow the other way? They still have the name, so one can only surmise that Ernst & Young stands behind their actions.
  - novaleaf 10y ago
    at a minimum, this seems like something the HK gov should get involved in......
- ProAm 10y ago
  Aren't most audits a joke anyhow? I know companies 'have' to do them but I've never been a part of a good or accurate one (not just E&Y either). Most auditors are just looking to CYA and get the paper signed.
joshstrange 10y ago
Can someone explain what the business reason for backdating certs is? I understand it was to get around the notAfter but I don't understand who gains from this? Is it just old code they are too cheap/lazy to update?
- heheocoenev 10y ago
  Yeah old java6 TLS stacks have problems with SHA256, amoung other legacy systems used by the old guard
- lathiat 10y ago
  To create SHA-1 certificates for compatibility with older software. SHA-1 certificates may not be issued after December 31, 2015 and in any case browsers will not trust them. They created new certificates in 2016, with a "Start Date" in 2015 so that they satisfied this rule. This is forbidden for two reasons, first you cannot "back date" certificates and second you are no longer allowed to issue SHA-1 certificates, so they back dated to work around this.

132 points by buovjaga 10y ago

WoSign's response[1] might also be of interest in this context. They seem to take, at least partially, ownership of the issues. Am I also correct in that this only affects intermediate certificates? I see they say that they will have a workaround in place in about a months time. [1]: https://www.wosign.com/English/News/announcement_about_Mozilla_Action_20161024.htm https://www.wosign.com/English/News/announcement_about_Mozil...

jlgaddis 10y ago
> Am I also correct in that this only affects intermediate certificates? No. As the article states, it will affect any certificates that chain up to the specified root certificates (including and intermediate and end-entity certificates). > I see they say that they will have a workaround in place ... From previous statements, I believe WoSign's plan is to resell another CA's certificates during the period that they don't have a root of their own in the trust store.
- pilif 10y ago
  > From previous statements, I believe WoSign's plan is to resell another CA's certificates during the period that they don't have a root of their own in the trust store. or find/buy another currently trusted CA that cross-signs their new root. I'm sure they can find somebody. edit: yes. that's what they are going to do according to https://www.wosign.com/English/News/announcement_about_Mozilla_Action_20161024.htm https://www.wosign.com/English/News/announcement_about_Mozil...: > There will be new SSL certificates issued by a new WoSign intermediate CA which is signed by the one of global trusted root CA, it supports all the browsers (including Firefox). This will be done within one months. I wonder who's going to be stupid/reckless enough to sign that intermediate.
  - tankenmate 10y ago
    Indeed, any company that signs any of their intermediates would want to have an ironclad indemnification, penalty clauses and a fully paid up escrow (hosted in a neutral country like Switzerland, Ireland or the like) in any contract. But I suspect that another Chinese company or Chinese owned company will be selected to be the signor.
  - pfg 10y ago
    It remains to be seen whether they will actually hold the private key for that intermediate certificate and issue end-entity certificates from it, or if this is just some sort of reselling deal where a different, trusted CA holds the key, performs domain validation, etc (which is a fairly common practice). I have my doubts about whether Mozilla will accept them continuing to operate an actual CA with a new cross-signed certificate prior to them completing the inclusion process. CAs need to disclose these intermediate certificates, and I expect it would end up being revoked, with possible sanctions for whoever cross-signs them.
  - bandrami 10y ago
    I wonder who's going to be stupid/reckless enough to sign that intermediate. You have a much higher opinion of the probity (and competence) of CAs than is probably warranted.
TravelTechGuy 10y ago
I've noticed 2 things in their response: 1. They take no direct responsibility, nor try to explain, or excuse the deception claims. To WoSign, the whole thing is "an incident", not a premeditated deception. 2. They still do not acknowledge their ownership of StartCom, or explain why that was kept in the dark. In fact they keep talking about "4 WoSign roots" despite the 6 roots mentioned by Mozilla.

jo909 10y ago

I am kind of thankful for this. It gave us immediate time budget to switch our certificate process over to Letsencrypt. I'm also glad it does not affect existing certs, since we need some weeks to ramp up the LE certs because of the 20 new certs per week limit (we can continue to only bundle subdomains per cert that are actually used on the same host/loadbalancer).

emilburzo 10y ago
Do you have multiple hosts serving the same domain by any chance? I'm wondering if there's been any progress in finding an easy solution for this use-case.
- pilif 10y ago
  Many if not all of the acme clients out there store the key and certificate as a file. Just copy that to the various machines terminating the SSL connection. In our case, there's a script on the frontend machine running HAProxy that's fetching the certificates and putting them on a shared file system so the backup machine also has access to them. The traffic between frontend host and application servers is unencrypted in our case as it's all one rack under our control. If it wasn't, we'd be using a self-signed certificate for that connection.

0x0 10y ago

Interestingly, over at the Google Certificate Transparency group, it looks like both StartCom's and WoSign's CT log servers were recently included into Chrome. https://groups.google.com/a/chromium.org/forum/#!topic/ct-policy/4BNen-sogf8 https://groups.google.com/a/chromium.org/forum/#!topic/ct-po...

iancarroll 10y ago
Luckily CT logs are designed to be mostly trustless.
- 0x0 10y ago
  Yet the Izenpe CT log managed to be untrustworthy enough to be de-listed! https://groups.google.com/a/chromium.org/forum/#!topic/ct-policy/g0WWgUodpOk https://groups.google.com/a/chromium.org/forum/#!topic/ct-po...
  - iancarroll 10y ago
    Indeed. For those curious, CT is designed to "discover" SCTs that were not publicly shown by the log, so that the log can operate without a lot of trust, though this did not happen here (someone just found them operating two logs...) I guess I'm not the best person to criticize other logs, though. :-)
dorianm 10y ago
"Please note that while both Symantec and CNNIC's logs are presently enabled in the current Chrome stable release (Chrome 52), both WoSign and StartCom's logs will not be recognized until Chrome 54, which is presently the Dev channel release."
- randomfool 10y ago
  https://googlechromereleases.blogspot.com/2016/10/stable-channel-update-for-desktop_20.html https://googlechromereleases.blogspot.com/2016/10/stable-cha... Chrome 54 is now in stable.

jpablo 10y ago

I don't like this at all. The only value left in WoSign root certificates is in issuing backdated certificates that wouldn't be widely distributed. So basically the only way to extract any money out of their current root keys is to sell rogue certificates for targeted attacks.

BillinghamJ 10y ago
Mozilla has explicitly stated that if they find any evidence of this happening, both WoSign and StartCom will be completely revoked - both for old and new certificates. > If additional back-dating is discovered (by any means) to circumvent this control, then Mozilla will immediately and permanently revoke trust in the affected roots.
- jamiesonbecker 10y ago
  > if they find any evidence of this happening There's already evidence of this happening. It happened. Why does it have to happen again. Just revoke em.
  - inimino 10y ago
    The point of doing it this way is that it puts the CA out of business but doesn't break the Web, which makes it the only really practical way for browsers to exert leverage over too big to fail CA's.
    - lathiat 10y ago
      more accurately, this way all the existing issued certificates are not invalidated. the idea is not to punish the customers of wosign, startcom who already paid for a certificate and have it in use. if they removed it entirely, those would all break which would be inconvenient to the otherwise innocent customers.
      - jamiesonbecker 10y ago

lexman0 10y ago

"Although Mozilla’s sanctions are too severe..." These guys must be joking. Trust has been lost, the roots should be permanently revoked. If anything, I think Mozilla's actions are not severe enough. How likely is it that Mozilla doesn't know the full story? There may be additional violations that have been missed.

jcranmer 10y ago
Mozilla has revoked the CA certificate moving forward. Already-issued certificates are unaffected, to allow current customers time to migrate to a more reputable CA. If they were to immediately revoke the current roots, then thousands of sites would suddenly report certificate errors--which would train users to click through them, helping nobody. The axe Mozilla is holding over WoSign is the threat of immediate full revocation if WoSign is caught doing backdating again. Given that WoSign has been coerced into cooperating with publishing all CSRs via Certificate Transparency, and that there is likely to be a much larger group of people watching carefully for violations, I don't expect it to take very long for future backdated certificates to be caught if WoSign does try it.

bandrami 10y ago

Maybe I'm reading this wrong, but doesn't this mean that the actual, found-in-the-wild backdated certs will still be trusted? At any rate, this gets to the crux of PKI's problem. This was a backdating of certificates because of a change in policy about the cryptographic strength of hashes. But the weak point in PKI isn't the cryptography, it's the agents.

pfg 10y ago
IIRC they were revoked in the course of the investigation (somewhere in the last two months). That being said, the risky bit is that they actually signed those SHA-1 certificates in the first place - a certificate that could be specially crafted to "collide" with one for a site an attacker wants to impersonate. It's not really all that important whether they're revoked (though there's nothing wrong with revoking them just in case). As for CAs being the weak point, luckily we're going to be living in a mandatory Certificate Transparency world in the not-so-far future.
- bandrami 10y ago
  IDK, I'm worried this is an example of what that will look like: since the CAs are the actual stakeholders, the default question is going to be "what can we do to make this CA trustworthy again?" rather than "how can we make sure this CA never has another cert trusted again?"
jcranmer 10y ago
The backdated certificates that have already been found are explicitly being added to OneCRL, which is to say that they are explicitly distrusted independent of what the CRL/OSCP responder of WoSign and StartCom say.
- bandrami 10y ago
  So some portion of Firefox users will be protected from them. Better than nothing I guess.

jamiesonbecker 10y ago

It'd be nice if we could distrust the whole CA system and start from scratch. (of course, to everyone who is saying to themselves that this is absurd and unrealistic.. yes, of course you're right.. for the moment.) It seems pretty clear that the future lies in the blockchain.

dpark 10y ago
How does the blockchain solve the problems with CAs?
- pfg 10y ago
  Haven't you heard, you can solve any problem by throwing a blockchain at it! ;-) On a more serious note, Certificate Transparency, which will become mandatory in Chrome in about a year, uses technology that is strongly related to blockchains, so we're really not too far from a blockchain-like solution for the Web PKI.
  - Torgo 10y ago
    A global, permanent, public, append-only log for 11 cents a record is pretty useful.
  - schoen 10y ago
    I agree that CT uses a technology related to blockchains, but as I noted above, Ben Laurie has been pretty vocal in saying that the Bitcoin-style PoW blockchain isn't a technology that he favors, and I think he thinks CT is importantly different in some respects (for example in accepting more centralization in the operation of logs: an uninvolved anonymous party can't show up and "mine" a CT log event). http://www.links.org/files/decentralised-currencies.pdf http://www.links.org/files/decentralised-currencies.pdf
- throwaway7767 10y ago
  Not the OP, but I imagine he has something like namecoin in mind. Domains are tied to cryptographic keys which are also used for encryption+authentication, and the blockchain stores the mapping of names to keys. It's not ready for primetime, and it has its own issues. I don't really see it gaining much traction. But it does present an alternative approach to the problem.

JumpCrisscross 10y ago

> No longer accept audits carried out by Ernst & Young Hong Kong This is...a big deal? If Mozilla believes E&Y HK was negligent or complicit in WoSign's lies, then that newfound scepticism should extend to the office's financial audits.

throwaway7767 10y ago
Seems weird to tie it just to the Hong Kong office though. These guys are trading on the Ernst & Young name to get the positive reputation associated with it, why should the negative publicity not flow the other way? They still have the name, so one can only surmise that Ernst & Young stands behind their actions.
- novaleaf 10y ago
  at a minimum, this seems like something the HK gov should get involved in......
ProAm 10y ago
Aren't most audits a joke anyhow? I know companies 'have' to do them but I've never been a part of a good or accurate one (not just E&Y either). Most auditors are just looking to CYA and get the paper signed.

joshstrange 10y ago

Can someone explain what the business reason for backdating certs is? I understand it was to get around the notAfter but I don't understand who gains from this? Is it just old code they are too cheap/lazy to update?

heheocoenev 10y ago
Yeah old java6 TLS stacks have problems with SHA256, amoung other legacy systems used by the old guard
lathiat 10y ago
To create SHA-1 certificates for compatibility with older software. SHA-1 certificates may not be issued after December 31, 2015 and in any case browsers will not trust them. They created new certificates in 2016, with a "Start Date" in 2015 so that they satisfied this rule. This is forbidden for two reasons, first you cannot "back date" certificates and second you are no longer allowed to issue SHA-1 certificates, so they back dated to work around this.