9 ms·

DigiCert Statement on Trustico Certificate Revocation

163 points by el_duderino 9y ago

xwvvvvwx 9y ago
Had to read it like 3 times before I could process. Unreal.
- dullgiulio 9y ago
  Help me understand: this CEO has 20k private certificates he obviously should never have seen, yet alone stored. How is this not related to "the big distrust"? Unreal.
  - jackfraser 9y ago
    If they operate as any other CA does, there's no reason to assume that they would have had any way to have the private keys which would only ever be on customer systems. This implies that Trustico was sent the keys (or recieved them somehow) from a third party who had compromised them. 20k certs implies a tremendous number of clients; seems tome it's more likely one of Trustico's intermediate CA certs got compromised and these 20k are newly issues ones against that compromised CA cert.
    - praseodym 9y ago
      They apparently have a “certificate wizard” that will generate the certificate and corresponding private key for you. Of course that’s practical for the end user (generating a CSR can be cumbersome), but obviously insecure. They’re not the only party to do so either, e.g. DNSimple (which is otherwise a great company!) also provide APIs to have them generate and store certificates including private keys: https://developer.dnsimple.com/v2/certificates/#getCertificatePrivateKey https://developer.dnsimple.com/v2/certificates/#getCertifica...
      - bostik 9y ago
        > They apparently have a [webgui] “certificate wizard” that will generate the certificate and corresponding private key for you. Any company doing that deserves to have their decision makers crucified, head down, on the outer wall of the town church, with the nails hammered through their genitals. If you didn't create the private key yourself, by definition it's not private. If generating CRSs and handling the certificates is too difficult, then feed improvements upstream. And yes, I know how horrible openssl's UI is for dealing with anything other than CN based certs. It's awful enough for those, an absolutely atrocious for pretty much anything else. For the record, CN has been deprecated in certificates since ~2000 (RFC 2818), and practically obsoleted since 2011 (RFC 6125). Regardless of all this, I have dealt (in 2017) with parties who don't understand SAN over CN, and have even had their software stacks blow up when I provide a cert with SANs only, and no CN.
        ceejayoz 9y ago
        > If you didn't create the private key yourself, by definition it's not private. In fairness, though, your certificate vendor already has the power to issue a brand new certificate for your domain if they wanted to quietly snoop on traffic. Same thing with AWS generating SSH keys if you want them to - they can get at all the data on your server anyways. No, it's not the greatest practice, but it probably does help a bit for making things easier for less-technical users.
        tialaramex 9y ago
        Misconceptions at work: If a hypothetical Bad Guy has got a different Certificate for my site they cannot "quietly snoop on traffic". They have to do an active Man-in-the-middle attack on each connection or else it's completely opaque to them. Each of the clients they do this to receives a free Smoking Gun, a copy of their illegitimate certificate showing who signed it and when. In a future where CT policy enforcement is completed (not yet but maybe soon) they must also have logged the certificate so that everybody can see it, or it won't work. If the hypothetical Bad Guy has my Private Key, they can quietly snoop traffic using ciphersuites which lack "Forward Secrecy" although no popular browser will choose this unless forced to, and in TLS 1.3 Forward Secrecy is mandatory. If the Bad Guy has my Private Key and is willing to do a Man-in-the-middle attack the same applies as above, except that I don't learn anything from Certificate Transparency since they can use my cert as they know the key for it.
    - ceejayoz 9y ago
      In the thread at https://groups.google.com/forum/m/#!topic/mozilla.dev.security.policy/wxX4Yv0E3Mk https://groups.google.com/forum/m/#!topic/mozilla.dev.securi... is this tidbit: > We have purchased thousands of certificates using Trustico as a reseller within the last years. Back in these days Trustico created CSR / Private Key pair within their online platform (Yes, you read it right - you can create CSR/Private Key on their webpage !!!) which was the default at this time and it is still possible to do so in their web interface.
    - regecks 9y ago
      Well, Trustico is a reseller. There's very little to stop them doing anything they want, since they are not subject to CABF regulations. For example, consider this page: https://www.trustico.com.au/ssltools/create/csr-pem/create-a-new-csr-instantly.php https://www.trustico.com.au/ssltools/create/csr-pem/create-a... The best part - it doesn't even use browser crypto. The private key is generated server-side and then rendered in the server response. Private key generation on the reseller-side - what can go wrong.
      - BillinghamJ 9y ago
        They are still effectively subject to the CA/B rules. The issuer of the certs is subject to the rules, and they must ensure compliance of any companies they delegate to.
        pfg 9y ago
        This is not wrong, but in the context of them generating and storing private keys for their users it means there's nothing in the Baseline Requirements preventing them from doing so if the subscriber (user) agrees to it. However, even if we assume Trustico was an authorized party, the keys became compromised the moment they were disclosed to DigiCert unless the terms their users agreed to included DigiCert as an authorized party. Even if they were you could make an argument that those keys were compromised due to the fact that they were literally sent via email, unless they were properly encrypted and what not.
        mook 9y ago
        Given that they have access to private keys they shouldn't, should Comodo (their new CA partner) then think about stopping them from reselling? Obviously, they've already terminated their relationship with Symantec...
  - geofft 9y ago
    > How is this not related to "the big distrust"? Unreal. If you mean the Symantec distrust - it's financially related but not technically related. In particular, it looks like Trustico ended their business agreement with Symantec (over the Symantec distrust) and signed a new contract with Comodo to resell Comodo certs instead. They wanted to move all their customers to the Comodo certs, and asked Digicert, the new owners of the old Symantec root, to issue revocations. When Digicert said no, that's not how it works, they responded by sending over all of the private keys they had. The fact that they had those keys, and the fact that they sent them via email to Digicert, is entirely separate from any technical problem that Symantec or Digicert may have had.
    - dullgiulio 9y ago
      Thank you for your explanation. I remember part of the Symantec problem was the uncontrolled resellers practices. Isn't this just some more dust under the rag coming out now?
      - geofft 9y ago
        Probably. In some amount of fairness, Trustico's stated motivation was that they didn't feel like they trusted Symantec (reasonable!) and the same people were involved with the move to Digicert (which I think is correct, some employees moved but technical oversight should have moved to the more organizationally-competent Digicert team) and the same problems were likely to happen again (I think Digicert is generally good at being a competent CA, but it's not unreasonable for them to decide the risk is too high if some of the same people were around). They also state in the MDSP thread, "We were also a victim whereby Symantec mis-issued SSL Certificates owned by us, subsequently we were asked to keep the matter quiet, under a confidentially notice."
      - tialaramex 9y ago
        The big Symantec problem was RAs rather than resellers. The difference probably doesn't mean much to customers, but it means a lot in terms of trust. Symantec trusted the RAs to do Validation. So although we believe CrossCert (the Korean RA which made all this kick off) were actually making some sort of attempt to validate, since Symantec exercised no effective oversight and relied entirely upon third party auditors (whose role is _audit_ not actively overseeing everything) we can't be sure. We know CrossCert validated bogus certificates for example.com, which although it's scarcely google.com or a major bank is still very wrong. Executives at Symantec essentially did not do their job on this, that's why even if they hadn't quit the market voluntarily they were in the process of being forced to let somebody else do the actual oversight. Board-level incompetence is very widespread, but that's no reason we have to tolerate it in the Web PKI. Trustico was not trusted to validate things, so if they tried to sell some customer certificates for example.com, the customer would have to prove to Symantec that they legitimately controlled example.com to get their certificate. They could still cause (as seen here) mayhem, but only for their own customers, so arguably caveat emptor.
heywire 9y ago
"Trustico’s CEO indicated that Trustico held the private keys for those certificates, and then emailed us approximately 20,000 certificate private keys." Well, that's one way to ensure they're considered "comprimised".
- deleted 9y ago
  [deleted]
praseodym 9y ago
The situation is unfolding in this Mozilla security policy mailing list thread: https://groups.google.com/forum/m/#!topic/mozilla.dev.security.policy/wxX4Yv0E3Mk https://groups.google.com/forum/m/#!topic/mozilla.dev.securi...
- jaas 9y ago
  I hope it's not overly pedantic to point out that what you linked to is not a CAB forum thread. That's a thread on the Mozilla security policy list, which many root programs and CAs make use of for disclosure and discussion. CAB forum has its own lists elsewhere. A relevant difference is that CAB forum discusses creating and modifying policy but is not involved in enforcement. The root programs typically deal with enforcement.
  - praseodym 9y ago
    I stand corrected, thank you.
  - tialaramex 9y ago
    To elaborate slightly:: The CA/B is a standing meeting between two groups with very different objectives that have an ongoing need to reach some sort of agreement: the Certificate Authorities and the Browsers (in reality more or less the Operating System vendors except Mozilla represents the Free Unixes). This meeting must not appear to be a cartel because cartels are illegal, so it mustn't discuss prices or agree what products should or should not be marketed. m.d.s.policy is de facto the only public oversight for the entire Web PKI. Mozilla as a Browser vendor (and as hinted above, on behalf of all the Free Unixes) gets to insist upon Rules, not only enforcing rules agreed at CAB, but also making its own rules on top for Certificate Authorities in its Trust Store. As a Charity one of its rules is that it insists on doing almost everything in Public using m.d.s.policy, there is no other obligation on a Certificate Authority to engage with the public at all, but Mozilla's rules force them to tell Mozilla important things via this public group and to answer questions impertinent members of the public (like me) ask them on m.d.s.policy. Mozilla (and other Browser vendors) is entirely free to insist upon whatever rules they want. The CA/B does not bind them, like the UN it is a forum to meet, and to come to agreements, but it cannot force you to agree to anything you don't want. Two recent illustrative examples: The CA/B took ages to reform the Domain Validation methods, terribly weak methods had been popular for years, with all sorts of stupid design flaws, and after it did finally vote on a reformed list of methods, CA members claimed a bunch of patents needed to be licensed stalling things for months extra. Mozilla cut through the bullshit and said essentially "See that list of ten methods you voted on that's stuck in Lawyer hell? Too bad, our policy now requires you obey the list which we're naming the Ten Blessed Methods" Google wanted to reduce the maximum lifetime of new certificates. It takes ages to get CAs to introduce a new rule, and then years while the old certificates made under the old rule expire, too long for Google. So they made an ultimatum, fix it or we'll just tell Chrome to reject certificates after 90 days, and that's the new maximum certificate lifetime. The CA/B eventually voted through a compromise 825 days (which goes into effect tomorrow, 1 March 2018) instead of the previous 39 month limit.
albinowax_ 9y ago
This is insane. I guess Trustico emailed the private keys because Digicert was acting reluctant to revoke them? Pretty bold move. Hopefully the fact that revocation is largely broken doesn't make this worse...
deleted 9y ago
[deleted]
xwvvvvwx 9y ago
Apparently they had a web front end where customers could request private keys... https://twitter.com/GossiTheDog/status/968834765888589825 https://twitter.com/GossiTheDog/status/968834765888589825
- regecks 9y ago
  Had? Have. https://www.trustico.com.au/ssltools/create/csr-pem/create-a-new-csr-instantly.php https://www.trustico.com.au/ssltools/create/csr-pem/create-a... Generated server-side. http://termbin.com/eu4c http://termbin.com/eu4c
  - piracykills 9y ago
    Yeah, that's actually surprisingly common, I remember working with a few before that did it. StartSSL and a Comodo reseller of some sort I think. I'm done with these companies, Let's Encrypt seems to be the only people doing it right.
- stevekemp 9y ago
  And execute arbitrary commands as root: https://twitter.com/Manawyrm/status/969230542578348033 https://twitter.com/Manawyrm/status/969230542578348033
smpetrey 9y ago
Fucking insanity.
QML 9y ago
I recently learned in my security class that certificates are the way by which domains are publicly identified. But I don’t understand why there hasn’t been any alternatives besides trusting these companies to issue certificates...
- SteveNuts 9y ago
  It would be really hard to keep everyone's browsers/operating systems updated with a list of certificates as they're generated and renewed. You pretty much have to concentrate it down to a small subset of trusted certificate authorities to sign your certificate. Also, it should be a way to verify that random people aren't able to get certificates on behalf of a domain they don't own (there's nothing stopping me from generating a cert for google.com - but it should be impossible for me to get it trusted by your browser).
an_account_name 9y ago
I won't be too surprised if Comodo decides not to do business with Trustico after this.
acd 9y ago
Interesting move by the CEO. Personally thinks that centralized CA proving trust of unknowns of long distance again and again prove that the security of centralized trust is weak!
weiland 9y ago
First of all who is trustico. Second of all why do SSL certs need to be resold?
0x0 9y ago
The DV CA space is such a mess. I still wish DV certificate issuance was the responsibility of the domain registrar, with technical measures taken to prevent competing registrars from misissuance. This way, you don't need to play whack-a-mole around your certificates, just pick a reputable registrar when you buy your domain, avoid the need to "prove" you hold the domain (your registrar control panel is today already the sole security barrier protecting your domain), and avoid the possibility that a third party can play the impostor game. And it would bring some meaning back to the reputation of different TLDs and their various registrars.
- brightball 9y ago
  CAA records help with that now. You can just specify your certificate issuer at the DNS level.
  - deleted 9y ago
    [deleted]
  - 0x0 9y ago
    See, that requires trusting that all CAs actually respect the CAA records. AFAIK, CAA records aren't validated by end-user client software during SSL negotiation. I would love to see some technical measure implemented where DV SSL certificate issuance and validation was tied to the TLD registry and the domain's particular registrar. It would eliminate the frankly unnecessary trust we put in all root CAs and their sub-CAs to behave. (Well, at least CT logs are becoming a thing now) It would also eliminate today's more or less awkward validation routines that depend on third parties querying DNS/HTTP/whois-email addresses. The registry has the definite database, mapping the domain to your account at your registrar, anyways. They should simply have a public key upload button next to your domain name in your registrar's domain name control panel that immediately returns an appropriate certificate, no further validation needed :)
    - zokier 9y ago
      > I would love to see some technical measure implemented where DV SSL certificate issuance and validation was tied to the TLD registry and the domain's particular registrar Tying PKI to DNS veers pretty close to DANE, and I imagine most of the criticism against DANE would remain valid.
pfg 9y ago
Trustico now posted a statement on their site[1]. My favourite part is this: > Trustico® followed the requests of DigiCert by initially recovering Private Keys from cold storage and subsequently e-mailing the associated order number and Private Keys to DigiCert in a ZIP file. The file did not contain any other type of data. [1]: https://www.trustico.com/news/2018/symantec-revocation/certificate-replacement.php https://www.trustico.com/news/2018/symantec-revocation/certi...
- Lazare 9y ago
  What a bizarre defense. Do they think this makes them look...better? The core issue is 1) they had the keys at all 2) they decided to compromise them by emailing them DigiCert Saying "hey, we did nothing wrong, all we did is email the keys we had to DigiCert" just isn't a very compelling defense to claims they had the keys and emailed them to DigiCert. More like a "total admission of guilt" than a defense, really.
  - lathiat 9y ago
    The secondary part is that they requested mass revocation of active certificates, including from brands such as RapidSSL which were not being browser distrusted AFAIK (even though some others were.. if I need to be corrected on this please let me know). They did this without notifying their customers in advance, so literally, they were going to intentionally screw their customers by revoking all their active SSL certificates without their authorisation and without any notice. Which then happened. This would have affected ALL certificates, but since Digicert said no, they then pulled all of these generated private keys from storage which they should NOT have been storing -- on their reseller web interface when you generate a certificate (which is a questionable activity but lets just go with it for the argument).. a few days after you place the order the private key disappears from the web interface. You would therefor assume they deleted it. It seems they kept a copy of all of them this whole time - which is doubly irresponsible let alone stupid. All I can say is thankfully LetsEncrypt is making these guys mostly irrelevant... The good news is, as far as I know, they're only revoking the certificates that used the generator at least (though I haven't confirmed that) -- so hopefully most people that used a normal CSR certificate won't have them revoked. Bunch of morons, I can't see why they thought this was a good idea, and as the commentator above said, this response doesn't sound better in any way. It just proves they are intentionally screwing their customers for no real reason. Even if they were intent on this distrust process, they should have at least been ensuring customers got new certificates first and had plenty of notice -- e.g. how the chrome distrust worked. They didn't.
ggm 9y ago
Nobody who has depth in the business would create keypairs online if they could avoid it. So I ask myself if key escrow requirements "back then" permitted this kind of engagement with an agent, on behalf of government or other constrained PKI contexts where escrow was a requirement. If its just "for your convenience" I would have walked off this keypair years ago. Private key generated for me one line and held in your keystore? No thank you.
deleted 9y ago
[deleted]
jrochkind1 9y ago
I do not understand.
- deleted 9y ago
  [deleted]
olliej 9y ago
To clarify trustico disclosed private keys to an entity other than the owner of the key, which automatically starts a 24 hour countdown for the CA. regardless of who discloses the private key, a CA is required to revoke any keys that they learn have been compromised within 24 hours. CAs have been reprimanded in the past for failing to do so. The only ambiguity is the initial request to revoke by trustico - essentially for a CA the question is whether trustico is the agent for the certificatesTo clarify they disclosed private keys to an entity other than the owner of the key, which automatically starts a 24 hour countdown for the CA. The only ambiguity here is whether a reseller like trustico gets to decide to revoke certificates that they sold or whether the end users are the owner and so are the only people who can request revocation without proof of key compromise. From the point of view of the CA the initial “revoke all these certs “ email could have been any random person on the internet collecting public certificates, and then telling the CA to revoke them. The communication that needlessly included the private keys was equivalent to someone trawling github or what have you, then reporting all private keys that they found to the appropriate CA. It doesn’t matter how someone proves access to the private key, and it doesn’t matter who demonstrates it, all that matters is that someone said “here is evidence this key is no longer under complete control by its owner”. At that point a CA must revoke and no one gets to stop that.
dwd 9y ago
Having receiving multiple emails and free certificate replacements from all parties involved, it's really just a turf war. Trustico want to sell their customers their own branded certs, and DigiCert aren't going to just let those customers go. I for one am done with all of them and had any affected sites shifted to auto-renew certs within about 20 min.
rzr999 9y ago
On a side note they have a shady business model. When it comes to using certificates regularly they withhold funds if you don't buy enough certificates from them until you top up with 200 USD again. That means funds just get stuck. They won't return them even if you cancel the account.
kentbrew 9y ago
Does anyone know the name of Trustico's CEO?
- kentbrew 9y ago
  Ah, never mind. Zane Lucas, who is listed as a "director" most places.

163 points by el_duderino 9y ago

xwvvvvwx 9y ago

Had to read it like 3 times before I could process. Unreal.

dullgiulio 9y ago
Help me understand: this CEO has 20k private certificates he obviously should never have seen, yet alone stored. How is this not related to "the big distrust"? Unreal.
- jackfraser 9y ago
  If they operate as any other CA does, there's no reason to assume that they would have had any way to have the private keys which would only ever be on customer systems. This implies that Trustico was sent the keys (or recieved them somehow) from a third party who had compromised them. 20k certs implies a tremendous number of clients; seems tome it's more likely one of Trustico's intermediate CA certs got compromised and these 20k are newly issues ones against that compromised CA cert.
  - praseodym 9y ago
    They apparently have a “certificate wizard” that will generate the certificate and corresponding private key for you. Of course that’s practical for the end user (generating a CSR can be cumbersome), but obviously insecure. They’re not the only party to do so either, e.g. DNSimple (which is otherwise a great company!) also provide APIs to have them generate and store certificates including private keys: https://developer.dnsimple.com/v2/certificates/#getCertificatePrivateKey https://developer.dnsimple.com/v2/certificates/#getCertifica...
    - bostik 9y ago
      > They apparently have a [webgui] “certificate wizard” that will generate the certificate and corresponding private key for you. Any company doing that deserves to have their decision makers crucified, head down, on the outer wall of the town church, with the nails hammered through their genitals. If you didn't create the private key yourself, by definition it's not private. If generating CRSs and handling the certificates is too difficult, then feed improvements upstream. And yes, I know how horrible openssl's UI is for dealing with anything other than CN based certs. It's awful enough for those, an absolutely atrocious for pretty much anything else. For the record, CN has been deprecated in certificates since ~2000 (RFC 2818), and practically obsoleted since 2011 (RFC 6125). Regardless of all this, I have dealt (in 2017) with parties who don't understand SAN over CN, and have even had their software stacks blow up when I provide a cert with SANs only, and no CN.
      - ceejayoz 9y ago
        > If you didn't create the private key yourself, by definition it's not private. In fairness, though, your certificate vendor already has the power to issue a brand new certificate for your domain if they wanted to quietly snoop on traffic. Same thing with AWS generating SSH keys if you want them to - they can get at all the data on your server anyways. No, it's not the greatest practice, but it probably does help a bit for making things easier for less-technical users.
        tialaramex 9y ago
        Misconceptions at work: If a hypothetical Bad Guy has got a different Certificate for my site they cannot "quietly snoop on traffic". They have to do an active Man-in-the-middle attack on each connection or else it's completely opaque to them. Each of the clients they do this to receives a free Smoking Gun, a copy of their illegitimate certificate showing who signed it and when. In a future where CT policy enforcement is completed (not yet but maybe soon) they must also have logged the certificate so that everybody can see it, or it won't work. If the hypothetical Bad Guy has my Private Key, they can quietly snoop traffic using ciphersuites which lack "Forward Secrecy" although no popular browser will choose this unless forced to, and in TLS 1.3 Forward Secrecy is mandatory. If the Bad Guy has my Private Key and is willing to do a Man-in-the-middle attack the same applies as above, except that I don't learn anything from Certificate Transparency since they can use my cert as they know the key for it.
  - ceejayoz 9y ago
    In the thread at https://groups.google.com/forum/m/#!topic/mozilla.dev.security.policy/wxX4Yv0E3Mk https://groups.google.com/forum/m/#!topic/mozilla.dev.securi... is this tidbit: > We have purchased thousands of certificates using Trustico as a reseller within the last years. Back in these days Trustico created CSR / Private Key pair within their online platform (Yes, you read it right - you can create CSR/Private Key on their webpage !!!) which was the default at this time and it is still possible to do so in their web interface.
  - regecks 9y ago
    Well, Trustico is a reseller. There's very little to stop them doing anything they want, since they are not subject to CABF regulations. For example, consider this page: https://www.trustico.com.au/ssltools/create/csr-pem/create-a-new-csr-instantly.php https://www.trustico.com.au/ssltools/create/csr-pem/create-a... The best part - it doesn't even use browser crypto. The private key is generated server-side and then rendered in the server response. Private key generation on the reseller-side - what can go wrong.
    - BillinghamJ 9y ago
      They are still effectively subject to the CA/B rules. The issuer of the certs is subject to the rules, and they must ensure compliance of any companies they delegate to.
      - pfg 9y ago
        This is not wrong, but in the context of them generating and storing private keys for their users it means there's nothing in the Baseline Requirements preventing them from doing so if the subscriber (user) agrees to it. However, even if we assume Trustico was an authorized party, the keys became compromised the moment they were disclosed to DigiCert unless the terms their users agreed to included DigiCert as an authorized party. Even if they were you could make an argument that those keys were compromised due to the fact that they were literally sent via email, unless they were properly encrypted and what not.
      - mook 9y ago
        Given that they have access to private keys they shouldn't, should Comodo (their new CA partner) then think about stopping them from reselling? Obviously, they've already terminated their relationship with Symantec...
- geofft 9y ago
  > How is this not related to "the big distrust"? Unreal. If you mean the Symantec distrust - it's financially related but not technically related. In particular, it looks like Trustico ended their business agreement with Symantec (over the Symantec distrust) and signed a new contract with Comodo to resell Comodo certs instead. They wanted to move all their customers to the Comodo certs, and asked Digicert, the new owners of the old Symantec root, to issue revocations. When Digicert said no, that's not how it works, they responded by sending over all of the private keys they had. The fact that they had those keys, and the fact that they sent them via email to Digicert, is entirely separate from any technical problem that Symantec or Digicert may have had.
  - dullgiulio 9y ago
    Thank you for your explanation. I remember part of the Symantec problem was the uncontrolled resellers practices. Isn't this just some more dust under the rag coming out now?
    - geofft 9y ago
      Probably. In some amount of fairness, Trustico's stated motivation was that they didn't feel like they trusted Symantec (reasonable!) and the same people were involved with the move to Digicert (which I think is correct, some employees moved but technical oversight should have moved to the more organizationally-competent Digicert team) and the same problems were likely to happen again (I think Digicert is generally good at being a competent CA, but it's not unreasonable for them to decide the risk is too high if some of the same people were around). They also state in the MDSP thread, "We were also a victim whereby Symantec mis-issued SSL Certificates owned by us, subsequently we were asked to keep the matter quiet, under a confidentially notice."
    - tialaramex 9y ago
      The big Symantec problem was RAs rather than resellers. The difference probably doesn't mean much to customers, but it means a lot in terms of trust. Symantec trusted the RAs to do Validation. So although we believe CrossCert (the Korean RA which made all this kick off) were actually making some sort of attempt to validate, since Symantec exercised no effective oversight and relied entirely upon third party auditors (whose role is _audit_ not actively overseeing everything) we can't be sure. We know CrossCert validated bogus certificates for example.com, which although it's scarcely google.com or a major bank is still very wrong. Executives at Symantec essentially did not do their job on this, that's why even if they hadn't quit the market voluntarily they were in the process of being forced to let somebody else do the actual oversight. Board-level incompetence is very widespread, but that's no reason we have to tolerate it in the Web PKI. Trustico was not trusted to validate things, so if they tried to sell some customer certificates for example.com, the customer would have to prove to Symantec that they legitimately controlled example.com to get their certificate. They could still cause (as seen here) mayhem, but only for their own customers, so arguably caveat emptor.

heywire 9y ago

"Trustico’s CEO indicated that Trustico held the private keys for those certificates, and then emailed us approximately 20,000 certificate private keys." Well, that's one way to ensure they're considered "comprimised".

deleted 9y ago
[deleted]

praseodym 9y ago

The situation is unfolding in this Mozilla security policy mailing list thread: https://groups.google.com/forum/m/#!topic/mozilla.dev.security.policy/wxX4Yv0E3Mk https://groups.google.com/forum/m/#!topic/mozilla.dev.securi...

jaas 9y ago
I hope it's not overly pedantic to point out that what you linked to is not a CAB forum thread. That's a thread on the Mozilla security policy list, which many root programs and CAs make use of for disclosure and discussion. CAB forum has its own lists elsewhere. A relevant difference is that CAB forum discusses creating and modifying policy but is not involved in enforcement. The root programs typically deal with enforcement.
- praseodym 9y ago
  I stand corrected, thank you.
- tialaramex 9y ago
  To elaborate slightly:: The CA/B is a standing meeting between two groups with very different objectives that have an ongoing need to reach some sort of agreement: the Certificate Authorities and the Browsers (in reality more or less the Operating System vendors except Mozilla represents the Free Unixes). This meeting must not appear to be a cartel because cartels are illegal, so it mustn't discuss prices or agree what products should or should not be marketed. m.d.s.policy is de facto the only public oversight for the entire Web PKI. Mozilla as a Browser vendor (and as hinted above, on behalf of all the Free Unixes) gets to insist upon Rules, not only enforcing rules agreed at CAB, but also making its own rules on top for Certificate Authorities in its Trust Store. As a Charity one of its rules is that it insists on doing almost everything in Public using m.d.s.policy, there is no other obligation on a Certificate Authority to engage with the public at all, but Mozilla's rules force them to tell Mozilla important things via this public group and to answer questions impertinent members of the public (like me) ask them on m.d.s.policy. Mozilla (and other Browser vendors) is entirely free to insist upon whatever rules they want. The CA/B does not bind them, like the UN it is a forum to meet, and to come to agreements, but it cannot force you to agree to anything you don't want. Two recent illustrative examples: The CA/B took ages to reform the Domain Validation methods, terribly weak methods had been popular for years, with all sorts of stupid design flaws, and after it did finally vote on a reformed list of methods, CA members claimed a bunch of patents needed to be licensed stalling things for months extra. Mozilla cut through the bullshit and said essentially "See that list of ten methods you voted on that's stuck in Lawyer hell? Too bad, our policy now requires you obey the list which we're naming the Ten Blessed Methods" Google wanted to reduce the maximum lifetime of new certificates. It takes ages to get CAs to introduce a new rule, and then years while the old certificates made under the old rule expire, too long for Google. So they made an ultimatum, fix it or we'll just tell Chrome to reject certificates after 90 days, and that's the new maximum certificate lifetime. The CA/B eventually voted through a compromise 825 days (which goes into effect tomorrow, 1 March 2018) instead of the previous 39 month limit.

albinowax_ 9y ago

This is insane. I guess Trustico emailed the private keys because Digicert was acting reluctant to revoke them? Pretty bold move. Hopefully the fact that revocation is largely broken doesn't make this worse...

deleted 9y ago

[deleted]

xwvvvvwx 9y ago

Apparently they had a web front end where customers could request private keys... https://twitter.com/GossiTheDog/status/968834765888589825 https://twitter.com/GossiTheDog/status/968834765888589825

regecks 9y ago
Had? Have. https://www.trustico.com.au/ssltools/create/csr-pem/create-a-new-csr-instantly.php https://www.trustico.com.au/ssltools/create/csr-pem/create-a... Generated server-side. http://termbin.com/eu4c http://termbin.com/eu4c
- piracykills 9y ago
  Yeah, that's actually surprisingly common, I remember working with a few before that did it. StartSSL and a Comodo reseller of some sort I think. I'm done with these companies, Let's Encrypt seems to be the only people doing it right.
stevekemp 9y ago
And execute arbitrary commands as root: https://twitter.com/Manawyrm/status/969230542578348033 https://twitter.com/Manawyrm/status/969230542578348033

smpetrey 9y ago

Fucking insanity.

QML 9y ago

I recently learned in my security class that certificates are the way by which domains are publicly identified. But I don’t understand why there hasn’t been any alternatives besides trusting these companies to issue certificates...

SteveNuts 9y ago
It would be really hard to keep everyone's browsers/operating systems updated with a list of certificates as they're generated and renewed. You pretty much have to concentrate it down to a small subset of trusted certificate authorities to sign your certificate. Also, it should be a way to verify that random people aren't able to get certificates on behalf of a domain they don't own (there's nothing stopping me from generating a cert for google.com - but it should be impossible for me to get it trusted by your browser).

an_account_name 9y ago

I won't be too surprised if Comodo decides not to do business with Trustico after this.

acd 9y ago

Interesting move by the CEO. Personally thinks that centralized CA proving trust of unknowns of long distance again and again prove that the security of centralized trust is weak!

weiland 9y ago

First of all who is trustico. Second of all why do SSL certs need to be resold?

0x0 9y ago

The DV CA space is such a mess. I still wish DV certificate issuance was the responsibility of the domain registrar, with technical measures taken to prevent competing registrars from misissuance. This way, you don't need to play whack-a-mole around your certificates, just pick a reputable registrar when you buy your domain, avoid the need to "prove" you hold the domain (your registrar control panel is today already the sole security barrier protecting your domain), and avoid the possibility that a third party can play the impostor game. And it would bring some meaning back to the reputation of different TLDs and their various registrars.

brightball 9y ago
CAA records help with that now. You can just specify your certificate issuer at the DNS level.
- deleted 9y ago
  [deleted]
- 0x0 9y ago
  See, that requires trusting that all CAs actually respect the CAA records. AFAIK, CAA records aren't validated by end-user client software during SSL negotiation. I would love to see some technical measure implemented where DV SSL certificate issuance and validation was tied to the TLD registry and the domain's particular registrar. It would eliminate the frankly unnecessary trust we put in all root CAs and their sub-CAs to behave. (Well, at least CT logs are becoming a thing now) It would also eliminate today's more or less awkward validation routines that depend on third parties querying DNS/HTTP/whois-email addresses. The registry has the definite database, mapping the domain to your account at your registrar, anyways. They should simply have a public key upload button next to your domain name in your registrar's domain name control panel that immediately returns an appropriate certificate, no further validation needed :)
  - zokier 9y ago
    > I would love to see some technical measure implemented where DV SSL certificate issuance and validation was tied to the TLD registry and the domain's particular registrar Tying PKI to DNS veers pretty close to DANE, and I imagine most of the criticism against DANE would remain valid.

pfg 9y ago

Trustico now posted a statement on their site[1]. My favourite part is this: > Trustico® followed the requests of DigiCert by initially recovering Private Keys from cold storage and subsequently e-mailing the associated order number and Private Keys to DigiCert in a ZIP file. The file did not contain any other type of data. [1]: https://www.trustico.com/news/2018/symantec-revocation/certificate-replacement.php https://www.trustico.com/news/2018/symantec-revocation/certi...

Lazare 9y ago
What a bizarre defense. Do they think this makes them look...better? The core issue is 1) they had the keys at all 2) they decided to compromise them by emailing them DigiCert Saying "hey, we did nothing wrong, all we did is email the keys we had to DigiCert" just isn't a very compelling defense to claims they had the keys and emailed them to DigiCert. More like a "total admission of guilt" than a defense, really.
- lathiat 9y ago
  The secondary part is that they requested mass revocation of active certificates, including from brands such as RapidSSL which were not being browser distrusted AFAIK (even though some others were.. if I need to be corrected on this please let me know). They did this without notifying their customers in advance, so literally, they were going to intentionally screw their customers by revoking all their active SSL certificates without their authorisation and without any notice. Which then happened. This would have affected ALL certificates, but since Digicert said no, they then pulled all of these generated private keys from storage which they should NOT have been storing -- on their reseller web interface when you generate a certificate (which is a questionable activity but lets just go with it for the argument).. a few days after you place the order the private key disappears from the web interface. You would therefor assume they deleted it. It seems they kept a copy of all of them this whole time - which is doubly irresponsible let alone stupid. All I can say is thankfully LetsEncrypt is making these guys mostly irrelevant... The good news is, as far as I know, they're only revoking the certificates that used the generator at least (though I haven't confirmed that) -- so hopefully most people that used a normal CSR certificate won't have them revoked. Bunch of morons, I can't see why they thought this was a good idea, and as the commentator above said, this response doesn't sound better in any way. It just proves they are intentionally screwing their customers for no real reason. Even if they were intent on this distrust process, they should have at least been ensuring customers got new certificates first and had plenty of notice -- e.g. how the chrome distrust worked. They didn't.

ggm 9y ago

Nobody who has depth in the business would create keypairs online if they could avoid it. So I ask myself if key escrow requirements "back then" permitted this kind of engagement with an agent, on behalf of government or other constrained PKI contexts where escrow was a requirement. If its just "for your convenience" I would have walked off this keypair years ago. Private key generated for me one line and held in your keystore? No thank you.

deleted 9y ago

[deleted]

jrochkind1 9y ago

I do not understand.

deleted 9y ago
[deleted]

olliej 9y ago

To clarify trustico disclosed private keys to an entity other than the owner of the key, which automatically starts a 24 hour countdown for the CA. regardless of who discloses the private key, a CA is required to revoke any keys that they learn have been compromised within 24 hours. CAs have been reprimanded in the past for failing to do so. The only ambiguity is the initial request to revoke by trustico - essentially for a CA the question is whether trustico is the agent for the certificatesTo clarify they disclosed private keys to an entity other than the owner of the key, which automatically starts a 24 hour countdown for the CA. The only ambiguity here is whether a reseller like trustico gets to decide to revoke certificates that they sold or whether the end users are the owner and so are the only people who can request revocation without proof of key compromise. From the point of view of the CA the initial “revoke all these certs “ email could have been any random person on the internet collecting public certificates, and then telling the CA to revoke them. The communication that needlessly included the private keys was equivalent to someone trawling github or what have you, then reporting all private keys that they found to the appropriate CA. It doesn’t matter how someone proves access to the private key, and it doesn’t matter who demonstrates it, all that matters is that someone said “here is evidence this key is no longer under complete control by its owner”. At that point a CA must revoke and no one gets to stop that.

dwd 9y ago

Having receiving multiple emails and free certificate replacements from all parties involved, it's really just a turf war. Trustico want to sell their customers their own branded certs, and DigiCert aren't going to just let those customers go. I for one am done with all of them and had any affected sites shifted to auto-renew certs within about 20 min.

rzr999 9y ago

On a side note they have a shady business model. When it comes to using certificates regularly they withhold funds if you don't buy enough certificates from them until you top up with 200 USD again. That means funds just get stuck. They won't return them even if you cancel the account.

kentbrew 9y ago

Does anyone know the name of Trustico's CEO?

kentbrew 9y ago
Ah, never mind. Zane Lucas, who is listed as a "director" most places.