14 ms·

Around 293 intermediate CAs in violation of CA/Browser guidelines

254 points by mbrinkers 6y ago

badrabbit 6y ago
Ironically, I get a HTTPS error when opening the link for this post. Edit: Now getting mac error: https://i.imgur.com/JmdC8Yi.png https://i.imgur.com/JmdC8Yi.png
- floatingatoll 6y ago
  Others aren’t reporting that. What error?
  - badrabbit 6y ago
    > Websites prove their identity via certificates. Firefox does not trust https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg13493.html https://www.mail-archive.com/dev-security-policy@lists.mozil... because its certificate issuer is unknown, the certificate is self-signed, or the server is not sending the correct intermediate certificates.
    - floatingatoll 6y ago
      The certificate is issued by Let’s Encrypt [2], and has a valid and correct intermediate chain from the server [1]. Have you knowingly altered your browser’s TLS security settings, or certificate root store settings (for example, to distrust X3), or are you running an especially old browser on an out-of-date platform? Being able to see a screenshot of which intermediate your browser is refusing to trust would be helpful [3]. (Unless you’re somehow being MITM’d, which can happen on some internet connections or with certain ‘security’ software on Windows or by mitmproxy left enabled, in which case the screenshot of the certificate chain will look nothing like Let’s Encrypt at all and help diagnose that too.) [1] Normal LE: https://www.ssllabs.com/ssltest/analyze.html?d=www.mail-archive.com https://www.ssllabs.com/ssltest/analyze.html?d=www.mail-arch... [2] Test site: https://valid-isrgrootx1.letsencrypt.org/ https://valid-isrgrootx1.letsencrypt.org/ [3] In the developer console, there should be a security tab with a View Details button.
      - badrabbit 6y ago
        I am using a mobile browser,it could be out of date but I didn't tinker with the network settings. I doubt only this site would get MITM'd. I will see if I can get more details from it. Screenshot: https://i.imgur.com/JmdC8Yi.png https://i.imgur.com/JmdC8Yi.png Now I get a MAC error insteaf of cert error
        floatingatoll 6y ago
        I would say with near certainty that your issues stem from your OS/browser, or if you have any security apps installed, those could be at fault too (since they sometimes run network interception). You might test a browser that ships its own SSL stack (I believe Firefox Android does, though I’m not 100% certain) and see if it Just Works in that, but at the end of the day, I’d simply recommend backing up your data and settings, factory resetting the device and updating it to latest, and then restoring your data and settings — there’s far too many things that can go wrong, especially in rooted scenarios, and I don’t have the ability to triage and repair beyond highlighting the three possible vectors you could tackle exploring yourself.
        badrabbit 6y ago
        Not rooted,no security software, tried different browsers and I am using FF android on that screenshot. Perhaps the stingrays are acting up this morning ;)
        floatingatoll 6y ago
        I wish I knew how to diagnose SSL issues in Firefox Android in order to learn more about why you’re experiencing issues here. If you’re on cellular, try WiFi? Does it affect any other phones in your house? Etc.
        tialaramex 6y ago
        The really labour intensive thing you could do goes like this: 1. Get a nice shiny modern Wireshark 2. Tell Firefox you want it to keep records of the session secrets that secure TLS. Set environment variable SSLKEYLOGFILE=/some/path/to/log/secret.keys 3. Packet capture the session you're interested in 4. Give Wireshark the packet capture (if not captured inside Wireshark itself) and the secret.keys 5. Now Wireshark can show the TLS session and you can see what went wrong in detail. So long as you didn't actually do anything secret you can give all these pieces to somebody else to look at. 6. Otherwise, after your investigation destroy the secret.keys and optionally the packet capture itself. I've used this level of effort to show a customer that, contrary to what they believed they were not presenting the nice client certificate I'd issued them when connecting. It turned out to be a config difference between their staging and production systems or something. But they were absolutely insistent their software was being turned away despite using a client cert (we used mutual TLS) so it took posting a Wireshark capture proving otherwise to get them to actually investigate.
plasma 6y ago
From the linked post: “ This is https://misissued.com/batch/138/ https://misissued.com/batch/138/ A quick inspection among the affected CAs include O fields of: QuoVadis, GlobalSign, Digicert, HARICA, Certinomis, AS Sertifitseeimiskeskus, Actalis, Atos, AC Camerfirma, SECOM, T-Systems, WISeKey, SCEE, and CNNIC.”
phit_ 6y ago
can anyone explain what that actually means for us endusers?
- Kliment 6y ago
  There are some entities that can sign certificates that don't conform to the standard the browser makers have agreed on. Those entities are about to have their signing rights revoked. Anything they signed will be invalidated, because there's no way to tell if the signature is legit or impersonated. If some sites are using certificates signed by those entities, they're going to have a bad time, and so will you as an end user of those sites. I'm not an expert in this, but this is the impact as far as I understand it. If I'm wrong and someone knows better, please correct me.
- floatingatoll 6y ago
  Q: Does this affect you? No action is otherwise required for end users, except for those site operators who manually deploy SSL certificates. If you manually deploy SSL certificates, then make sure that your contact information with the issuer (e.g. Digicert) is up-to-date; if your certificates are affected, they will contact you and advise you on how to proceed with reissuing and redeploying. Or, you can contact their customer service, reference the mail-archive link, and ask if your certificates are affected. If your SSL certificates are deployed automatically (Let’s Encrypt, AWS ACM, etc.) then no action is required. Either your certificates are unaffected, or they’ll be updated automatically by the automation. Q: What’s a simple summary of the problem? Imagine if root tried to userdel a malicious account, and the non-root user being deleted was able to tell the system “ignore that, I’m still valid”. The system would need to be quarantined and a replacement built without that flaw, since you could not state definitively that you had deleted the malicious user account.) This issue would allow a non-root certificate (‘intermediate’, ‘subCA’) to undo deletion (‘revocation’) of itself by the root authority, as well as undo deletion of its siblings (other intermediates issued by that root authority). To correct the issue, all non-root certificates issued incorrectly in this manner must be revoked and then destroyed, with proof of destruction. This ensures that the mis-issued intermediates can’t zombie-return someday by undoing their own deletion or the deletion of others.
sqldba 6y ago
Hilarious. And they’re referencing specs so deep nobody understands what they’re talking about. Certificates are BS.
- wrkronmiller 6y ago
  > I've flagged this as a SECURITY matter for CAs to carefully review, because in the cases where a third-party, other than the Issuing CA, operates such a certificate, the Issuing CA has delegated the ability to mint arbitrary OCSP responses to this third-party! > For example, consider this certificate https://crt.sh/?id=21606064 https://crt.sh/?id=21606064 . It was issued by DigiCert to Microsoft, granting Microsoft the ability to provide OCSP responses for any certificate issued by Digicert's Baltimore CyberTrust Root. We know from DigiCert's disclosures that this is independently operated by Microsoft.
- zeveb 6y ago
  > Certificates are BS. Pretty much. The whole business model never really made sense: the relying parties have no relationship with the certificate authorities, while the HTTPS servers are the customers of the CAs. I think it would make a lot more sense for certificates to be issued by domain owners, esp. since the original idea of tying sites to real-world businesses (e.g. with Dun & Bradstreet numbers) has been reduced to just verifying domain-name ownership. Edit: I think people misunderstand what I am saying here. What I mean is that I think that when one purchases a subdomain of domain, that domain should just issue a certificate — and that domain should only be allowed to issue certificates for its children. So e.g. if one purchases foo.com, then com issues a certificate for foo.com; if one purchases bar.net, then net issues a certificate for bar.net; if one purchases baz.ac.uk then ac.uk issued a certificate for baz.ac.uk. This is essentially what Let's Encrypt and ACME already do: com has the technical ability to reassign any of its subdomains at any time it wants to, and can get a certificate issued for any of them by reassigning & registering a certificate. And while we're at it, maybe we could kill ASN.1 with fire? Edit: if you downvoted for this, you have never tried to debug an ASN.1 BER file.
  - microcolonel 6y ago
    Then there's the whole issue of putting domain names/common names inside the certificates, but relying on external verification; rather than just having the DNS NICs directly sign for domains they have obvious authority over.
sp332 6y ago
"Within 7 days" counting from July 1?
- richardwhiuk 6y ago
  Yes
- pickledcods 6y ago
  Grab the popcorn
filleokus 6y ago
> Section 4.9.9 of the BRs requires that OCSP Delegated Responders MUST include an id-pkix-ocsp-nocheck extension. >For example, consider a certificate like https://crt.sh/?id=2657658699 https://crt.sh/?id=2657658699 . This certificate, from HARICA, meets Mozilla's definition of "Technically Constrained" for TLS, in that it lacks the id-kp-serverAuth EKU. However, because it includes the OCSP Signing EKU, this certificate can be used to sign arbitrary OCSP messages for HARICA's Root! >This also applies to non-technically-constrained sub-CAs. For example, consider this certificate https://crt.sh/?id=21606064 https://crt.sh/?id=21606064 . It was issued by DigiCert to Microsoft, granting Microsoft the ability to provide OCSP responses for any certificate issued by Digicert's Baltimore CyberTrust Root. We know from DigiCert's disclosures that this is independently operated by Microsoft. So my understanding is this: The CA's have issued certificates/sub-CA certs without the proper extension (or with too many extensions), causing those to be able to sign a OCSP response. And the Online Certificate Status Protocol (OSCP) is used to check the revocation status of certificates with the CA. So, this would allow e.g Microsoft to generate a fake OCSP response? That would perhaps be useful in some kind of MITM-attack scenario? While not good, perhaps not an end of the world problem either? However, I wonder how much problem will come for people needing to replace those soon to be revoked sub-CA certs...
- bluesign 6y ago
  If intermediate certificate private key leaks, basically you can block revocation. As you can sign OCSP message with this certificate. Kinda defeats purpose of revocation. But as far as I know, browsers are not failing hard on OCSP failure, if you can mitm the connection possibly you can block OCSP requests too.
- steventhedev 6y ago
  Many of the violations discussed on the security lists are not end of the world at all. The 63-bit instead of 64-bit serial number entropy issue is a good example of this. But the strict enforcement of all violations makes it easier to spot bad actors or at least those who aren't competently handling all of the requirements to be a CA. Bottom line: the entire CA system is built on trust. Would you trust someone who doesn't take issues seriously because they think they're small or unimportant? EDIT: reading the full report, it seems that the underlying risk is that if one of the intermediate CAs were to be compromised, even if it was revoked it could theoretically forge an OCSP response that it is still valid (and as a trusted CA issue certs for anything). So the response is very appropriate given the potential impact.
Mojah 6y ago
Shameless plug, but I built a monitoring solution with extensive SSL checks that’ll catch & report these if they’re revoked: https://ohdear.app https://ohdear.app
- agustif 6y ago
  This looks nice, I've recently started using instatus.com for the status page, but love your monitoring parts, might use the trial to further check it out!
deleted 6y ago
[deleted]
JakeWesorick 6y ago
What's an easy way to test if I am affected by this?
- floatingatoll 6y ago
  Contact the customer support of whoever issued your SSL certificates and ask if you are affected by this. If you use any of the major auto-issue, auto-renew certificate platforms, then you do not need to take any action — either you aren’t affected or they’ll issue a new certificate to you automatically. (Let’s Encrypt, AWS Certificate Manager, Google-Managed SSL Certificates, Heroku Automated Certificate Management)
- Mojah 6y ago
  Hi Jake! Enter your website URL in here, and it'll tell you if you're affected or not: https://ohdear.app/tools/certificate https://ohdear.app/tools/certificate
- tgsovlerkhgsel 6y ago
  Check if your browser shows a lock symbol on any important sites you visit. (This affects the PKI as a whole, because a single unconstrained compromised sub-CA can misissue for any domain - so if you use HTTPS, you're affected). If your question is whether you should take action, the answer is no, unless you're in some way responsible for an intermediate CA, which is something you'd know about.
zenexer 6y ago
The link is very technical, resulting in some confusion as to why this is such a big problem. The comments on HN reflect that. Here’s my understanding: This isn’t a problem because a sub-CA can revoke any certificate from any other sub-CA of the same CA. That would be bad, but, at worst, it’s denial-of-service. Rather, this is a problem because any sub-CA can effectively reverse the revocation of any other sub-CA, or the CA itself. That’s immensely problematic. Suddenly, the CA has no reliable way fully revoke certificates. Revocation is already somewhat broken as it is, but this gives a lot of entities the ability to deliberately interfere with revocation in ways that they shouldn’t be able to. The author goes on to explain that revocation of the affected certificates is insufficient, because they could be used to effectively reverse their own revocation at any point in the future. Instead, it must be proven that all copies of the keys have been destroyed. That’s quite an undertaking. What the author fails to mention is that revocation is already pretty broken. Most major browsers have their own built-in CRL replacements that contain the most important revocations they need to know about. Some browsers, like Firefox, may make additional efforts to ensure that any given certificate hasn’t been revoked; others, like Chrome, don’t. If you’ve ever visited a site that gives you a certificate error in Firefox but not Chrome, that’s likely why. In the case of browsers, it should be possible for each browser to forcefully revoke affected certificates, but revoking a sub-CA certificate is quite disruptive, so I’d be surprised if that happens within 7 days. The catch is that this technique is really only effective in modern, up-to-date browsers. In any case, the title is misleading. I don’t see where the author guarantees that this will happen within 7 days. The author claims it should happen within 7 days, but considering that the damage is already done and cannot be fully reversed by revocation, I find it hard to believe enforcing that deadline makes sense here.
- zenexer 6y ago
  Flagging for a title change, as the revocation deadline was proposed by a single individual on a mailing list, and there’s no evidence from further discussion in that thread that a consensus has been reached. I would not expect all these certificates to be revoked within 7 days.
  - mbrinkers 6y ago
yread 6y ago
Wow https://crt.sh/?id=7846179 https://crt.sh/?id=7846179 is also on the list, the certificate used by Dutch government to authenticate citizens. Yikes.
floatingatoll 6y ago
Digicert’s reply is called out by the original poster as being particularly high quality: https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg13535.html https://www.mail-archive.com/dev-security-policy@lists.mozil...
terom 6y ago
There's some related technical background in this 2014 bug report on Firefox rejecting a TLS server certificate issued by such an intermediate CA with the problematic 'OCSP Signing' EKU: https://bugzilla.mozilla.org/show_bug.cgi?id=991209#c7 https://bugzilla.mozilla.org/show_bug.cgi?id=991209#c7 The problem with intermediate CAs with the 'OCSP Signing' EKU being able to sign OCSP responses for any sibling certificates issued by the root CA seems to have been recognized for a while. But in this case, Mozilla allowed and ignored the 'OCSP Signing' EKU for vendor interop with existing intermediate CAs. Mozilla products would reject such OCSP responses, but the CA policy on issuing intermediate CA certificates with the 'OCSP Signing' EKU was not fixed. Specifically, MSADCS (Microsoft software used to issue certificates) requiring the CA certificate to have the 'OCSP Signing' EKU, "by design": https://support.microsoft.com/en-us/help/2962991/you-cannot-enroll-in-an-online-certificate-status-protocol-certificate https://support.microsoft.com/en-us/help/2962991/you-cannot-... The situation seems to be that the MSADCS policy on requiring the 'OCSP Signing' EKU on the intermediate CA certificate is incorrect, and clients validating OCSP responses signed by a delegated OCSP responder certificate do not require the 'OCSP Signing' EKUs on the issuing (intermediate/root) CA certificate(s): https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/XQd3rNF4yOo/bXYjt1mZAwAJ https://groups.google.com/forum/#!msg/mozilla.dev.security.p... This is in contrast to e.g. the TLS serverAuth EKU, where these EKUs chain: a technically constrained CA certificate must have a TLS serverAuth EKU in order to issue end-entity certificates with a TLS serverAuth EKU. The 'OCSP Signing' EKU on the intermediate CA certificate is unnecessary: the intermediate CA does not need the EKU to sign OCSP responses directly, nor to issue a delegated OCSP responder certificate with the 'OCSP Signing' EKU. The unnecessary 'OCSP Signing' EKU on the intermediate CA certificate is harmful, because it may be interpreted as a delegated OCSP responder certificate for the issuing root CA, capable of signing OCSP responses for any certificate issued by the root CA. The workaround for MSADCS is to use an untrusted CA certificate (e.g. self-signed) with the same public key and subject DN as the intermediate CA to issue the OCSP responder certificate, adding the 'OCSP Signing' EKU to both the OCSP responder certificate and the workaround-CA. The resulting OCSP responder certificate with the 'OCSP Signing' EKU will validate as an OCSP responder certificate for the actual intermediate CA, without including the 'OCSP Signing' EKU in the CA certificate. This BR policy violation approach seems to be an attempt to highlight the security problems and fix this long-standing issue, limiting the use of the 'OCSP Signing' EKU in order to protect clients from OCSP responses signed by intermediate CA certificates that were not intended to be delegated OCSP responder certificates for their root CAs.
dghughes 6y ago
Anyone remember two years ago when the Comodo CEO emailed hundred of private keys?
- tialaramex 6y ago
  That isn't actually what happened, but I don't doubt that's how you've remembered it. Comodo at this point controlled the CA roots that had belonged to Symantec. Trustico, a Symantec reseller (same sort of relationship to Symantec that your local Ford dealer has to the Ford motor company) asked Comodo to mass revoke thousands of certificates it had sold to third party subscribers as reseller for Symantec. It's not clear what Trustico hoped to achieve by that, maybe they believed they could get back the cost of the certificates? We don't know the details of the (confidential) contract between Trustico and Symantec or to what extent the contract terms survived transfer to Comodo. Maybe Trustico just wanted to push its customers into new deals, because it was not a Comodo reseller and risked being frozen out. Anyway, Jeremy Rowley, a Comodo VP asked for a reason to revoke these certificates, and by return he got thousands of private keys. Private Key compromise is a valid reason for revoking certificates, so Rowley confirmed the certificates matched these private keys and Comodo began revoking them. Trustico are the people who had thousands of private keys. A CA is strictly prohibited from having your private keys (and as we saw, if they are shown them they should revoke your certificates) but of course whether a CA enforces this rule on its resellers (via contract terms) is a matter between the CA and reseller. The whole point of private keys is that they're private. So, in one sense Trustico's customers got what they deserved - do not give your private keys to some reseller or trust them to pick keys for you. Nothing went wrong at Comodo here. And if you as a subscriber followed good practices you weren't affected either even if you'd bought certificates through Trustico. Only customers who'd gone with Trustico and done something inherently unsafe got burned.
  - sleevi 6y ago
    You said Comodo throughout, but it was DigiCert :)
    - tialaramex 6y ago
tptacek 6y ago
I was going to blog this after tormenting Sleevi on Twitter with more dumb questions, but I guess we're just going to do this here. The issue† --- it's super complicated in its particulars but not in its outline --- is that CAs have been issuing constrained intermediate CAs (CA's that can sign only a subset of certificates) that, because of a misconfiguration, can sign any OCSP message for their root CA. So, for instance, Sleevi points out a HARICA (Greek university CA) cert that is constrained by dint of not having the serverAuth EKU (so it can't sign TLS certs), but because it has the OCSPSigning EKU, can be used to sign OCSP messages. What happened was this: lots of CAs run by companies in the wild (remember: there are all sorts of constrained CAs running inside companies that you aren't supposed to have to care about, in part because they're constrained) run on Microsoft's CA software. Many of those CAs want to support OCSP. The way you generally set up OCSP is to sign a Delegated Responder certificate, which is an end-entity (non-CA) cert with the OCSPSigning EKU, which says "this certificate can be used to sign OCSP certs for this part of the PKI". But Microsoft's CA is broken: it won't let you sign a cert with the OCSPSigning EKU unless your CA cert also has that EKU (the EKUs must "chain"). So, that's what CAs did, despite the fact that chaining EKUs like that changed the semantics of what the certs were intended to express. So, as Sleevi put it, basically a bunch of CAs accidentally signed the OCSP equivalent of a bunch of CA:TRUE certificates. They gave their customers the ability to essentially disable revocation for the whole CA. We can go back and forth on how sound the WebPKI revocation infrastructure is with or without this mistake. But at a minimum we should be able to stipulate that strengthening revocation is an important project for browser vendors, and CAs that accidentally break the revocation infrastructure by misissuing certificates are an impediment to that project. There's an interesting backstory to this, which is what I actually wanted to write about, and that's CA/B Forum SC31. The CA/B Forum is the standards body that coordinates between browsers and CAs; they maintain the BRs, which are the bylaws for operating a CA that browsers will trust. The politics of CA/B Forum are weird, because CAs and browsers are structurally adversarial: browsers want maximal security regardless of the commercial implications for CAs (as they should). As a result of that weirdness, the browser vendors rely not only on the BRs, but also on their own bylaws for their respective root certificate programs. SC31 is an attempt by Ryan Sleevi to align the CA-approved BRs with the browser-approved root programs. Of course, the browser root programs, particularly Mozilla and Google's, are the only thing that really matters, because those rules determine whether the browsers themselves will honor a CA's certificates. But the CA/B Forum is dominated by CAs who sort of dispute this fact. So, SC31 asks that the BRs import that (now common) browser root program rule that certs can only live for just over a year. The "one year only" rule was considered previously by the CA/B Forum and failed in a vote (CA customers don't like that rule), so the CA's are unhappy to be relitigating that point. But then, the litigation itself is kind of theatrical, in that browsers already made this decision and mooted the debate. Standards bodies are weird. Anyways, what makes this funny is that, while the one-year restriction is the real problem that seemed to piss off the CAs in SC31, they also challenged some of its OCSP language. In the course of debating with them about whether SC31 described reality or not, Sleevi started looking at their OCSP issuances. And here we are: the CAs aren't even following their own BRs, and are breaking their bylaws in ways that materially impact TLS security. I understood literally none of this when I first read Ryan's message. I thought I knew some stuff about the WebPKI, but even with help on Twitter I had trouble with this eldritch stuff. I don't know how people like Sleevi do it, and, apparently, neither do many CA operators. This probably would have been a boring blog post anyways, but to make Kurt happy, I'll conclude with: "use Fly.io!". :) † Wrinkle: Sleevi flags these CA certs as missing the ocsp-nocheck attribute, which is presumably how he spotted them. ocsp-nocheck says "this certificate can't be trusted to OCSP revoke itself, because that's silly"; the idea is, certs flagged ocsp-nocheck rely on very short lifetimes rather than OCSP to manage revocation. Very short lifetimes are things you expect on end-entity certificates, which is what OCSP Delegated Responders are, but not so much on CA:TRUE certificates, which are what Sleevi actually found.
Mojah 6y ago
Here's a very quick way to check if you may be affected: this will check all certificates in the chain to see if they match the fingerprint of the to-be-revoked list. https://ohdear.app/tools/certificate https://ohdear.app/tools/certificate

254 points by mbrinkers 6y ago

Ironically, I get a HTTPS error when opening the link for this post. Edit: Now getting mac error: https://i.imgur.com/JmdC8Yi.png https://i.imgur.com/JmdC8Yi.png

floatingatoll 6y ago
Others aren’t reporting that. What error?
- badrabbit 6y ago
  > Websites prove their identity via certificates. Firefox does not trust https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg13493.html https://www.mail-archive.com/dev-security-policy@lists.mozil... because its certificate issuer is unknown, the certificate is self-signed, or the server is not sending the correct intermediate certificates.
  - floatingatoll 6y ago
    The certificate is issued by Let’s Encrypt [2], and has a valid and correct intermediate chain from the server [1]. Have you knowingly altered your browser’s TLS security settings, or certificate root store settings (for example, to distrust X3), or are you running an especially old browser on an out-of-date platform? Being able to see a screenshot of which intermediate your browser is refusing to trust would be helpful [3]. (Unless you’re somehow being MITM’d, which can happen on some internet connections or with certain ‘security’ software on Windows or by mitmproxy left enabled, in which case the screenshot of the certificate chain will look nothing like Let’s Encrypt at all and help diagnose that too.) [1] Normal LE: https://www.ssllabs.com/ssltest/analyze.html?d=www.mail-archive.com https://www.ssllabs.com/ssltest/analyze.html?d=www.mail-arch... [2] Test site: https://valid-isrgrootx1.letsencrypt.org/ https://valid-isrgrootx1.letsencrypt.org/ [3] In the developer console, there should be a security tab with a View Details button.
    - badrabbit 6y ago
      I am using a mobile browser,it could be out of date but I didn't tinker with the network settings. I doubt only this site would get MITM'd. I will see if I can get more details from it. Screenshot: https://i.imgur.com/JmdC8Yi.png https://i.imgur.com/JmdC8Yi.png Now I get a MAC error insteaf of cert error
      - floatingatoll 6y ago
        I would say with near certainty that your issues stem from your OS/browser, or if you have any security apps installed, those could be at fault too (since they sometimes run network interception). You might test a browser that ships its own SSL stack (I believe Firefox Android does, though I’m not 100% certain) and see if it Just Works in that, but at the end of the day, I’d simply recommend backing up your data and settings, factory resetting the device and updating it to latest, and then restoring your data and settings — there’s far too many things that can go wrong, especially in rooted scenarios, and I don’t have the ability to triage and repair beyond highlighting the three possible vectors you could tackle exploring yourself.
        badrabbit 6y ago
        Not rooted,no security software, tried different browsers and I am using FF android on that screenshot. Perhaps the stingrays are acting up this morning ;)
        floatingatoll 6y ago
        I wish I knew how to diagnose SSL issues in Firefox Android in order to learn more about why you’re experiencing issues here. If you’re on cellular, try WiFi? Does it affect any other phones in your house? Etc.
        tialaramex 6y ago
        The really labour intensive thing you could do goes like this: 1. Get a nice shiny modern Wireshark 2. Tell Firefox you want it to keep records of the session secrets that secure TLS. Set environment variable SSLKEYLOGFILE=/some/path/to/log/secret.keys 3. Packet capture the session you're interested in 4. Give Wireshark the packet capture (if not captured inside Wireshark itself) and the secret.keys 5. Now Wireshark can show the TLS session and you can see what went wrong in detail. So long as you didn't actually do anything secret you can give all these pieces to somebody else to look at. 6. Otherwise, after your investigation destroy the secret.keys and optionally the packet capture itself. I've used this level of effort to show a customer that, contrary to what they believed they were not presenting the nice client certificate I'd issued them when connecting. It turned out to be a config difference between their staging and production systems or something. But they were absolutely insistent their software was being turned away despite using a client cert (we used mutual TLS) so it took posting a Wireshark capture proving otherwise to get them to actually investigate.

plasma 6y ago

From the linked post: “ This is https://misissued.com/batch/138/ https://misissued.com/batch/138/ A quick inspection among the affected CAs include O fields of: QuoVadis, GlobalSign, Digicert, HARICA, Certinomis, AS Sertifitseeimiskeskus, Actalis, Atos, AC Camerfirma, SECOM, T-Systems, WISeKey, SCEE, and CNNIC.”

phit_ 6y ago

can anyone explain what that actually means for us endusers?

Kliment 6y ago
There are some entities that can sign certificates that don't conform to the standard the browser makers have agreed on. Those entities are about to have their signing rights revoked. Anything they signed will be invalidated, because there's no way to tell if the signature is legit or impersonated. If some sites are using certificates signed by those entities, they're going to have a bad time, and so will you as an end user of those sites. I'm not an expert in this, but this is the impact as far as I understand it. If I'm wrong and someone knows better, please correct me.
floatingatoll 6y ago
Q: Does this affect you? No action is otherwise required for end users, except for those site operators who manually deploy SSL certificates. If you manually deploy SSL certificates, then make sure that your contact information with the issuer (e.g. Digicert) is up-to-date; if your certificates are affected, they will contact you and advise you on how to proceed with reissuing and redeploying. Or, you can contact their customer service, reference the mail-archive link, and ask if your certificates are affected. If your SSL certificates are deployed automatically (Let’s Encrypt, AWS ACM, etc.) then no action is required. Either your certificates are unaffected, or they’ll be updated automatically by the automation. Q: What’s a simple summary of the problem? Imagine if root tried to userdel a malicious account, and the non-root user being deleted was able to tell the system “ignore that, I’m still valid”. The system would need to be quarantined and a replacement built without that flaw, since you could not state definitively that you had deleted the malicious user account.) This issue would allow a non-root certificate (‘intermediate’, ‘subCA’) to undo deletion (‘revocation’) of itself by the root authority, as well as undo deletion of its siblings (other intermediates issued by that root authority). To correct the issue, all non-root certificates issued incorrectly in this manner must be revoked and then destroyed, with proof of destruction. This ensures that the mis-issued intermediates can’t zombie-return someday by undoing their own deletion or the deletion of others.

sqldba 6y ago

Hilarious. And they’re referencing specs so deep nobody understands what they’re talking about. Certificates are BS.

wrkronmiller 6y ago
> I've flagged this as a SECURITY matter for CAs to carefully review, because in the cases where a third-party, other than the Issuing CA, operates such a certificate, the Issuing CA has delegated the ability to mint arbitrary OCSP responses to this third-party! > For example, consider this certificate https://crt.sh/?id=21606064 https://crt.sh/?id=21606064 . It was issued by DigiCert to Microsoft, granting Microsoft the ability to provide OCSP responses for any certificate issued by Digicert's Baltimore CyberTrust Root. We know from DigiCert's disclosures that this is independently operated by Microsoft.
zeveb 6y ago
> Certificates are BS. Pretty much. The whole business model never really made sense: the relying parties have no relationship with the certificate authorities, while the HTTPS servers are the customers of the CAs. I think it would make a lot more sense for certificates to be issued by domain owners, esp. since the original idea of tying sites to real-world businesses (e.g. with Dun & Bradstreet numbers) has been reduced to just verifying domain-name ownership. Edit: I think people misunderstand what I am saying here. What I mean is that I think that when one purchases a subdomain of domain, that domain should just issue a certificate — and that domain should only be allowed to issue certificates for its children. So e.g. if one purchases foo.com, then com issues a certificate for foo.com; if one purchases bar.net, then net issues a certificate for bar.net; if one purchases baz.ac.uk then ac.uk issued a certificate for baz.ac.uk. This is essentially what Let's Encrypt and ACME already do: com has the technical ability to reassign any of its subdomains at any time it wants to, and can get a certificate issued for any of them by reassigning & registering a certificate. And while we're at it, maybe we could kill ASN.1 with fire? Edit: if you downvoted for this, you have never tried to debug an ASN.1 BER file.
- microcolonel 6y ago
  Then there's the whole issue of putting domain names/common names inside the certificates, but relying on external verification; rather than just having the DNS NICs directly sign for domains they have obvious authority over.

sp332 6y ago

"Within 7 days" counting from July 1?

richardwhiuk 6y ago
Yes
pickledcods 6y ago
Grab the popcorn

filleokus 6y ago

> Section 4.9.9 of the BRs requires that OCSP Delegated Responders MUST include an id-pkix-ocsp-nocheck extension. >For example, consider a certificate like https://crt.sh/?id=2657658699 https://crt.sh/?id=2657658699 . This certificate, from HARICA, meets Mozilla's definition of "Technically Constrained" for TLS, in that it lacks the id-kp-serverAuth EKU. However, because it includes the OCSP Signing EKU, this certificate can be used to sign arbitrary OCSP messages for HARICA's Root! >This also applies to non-technically-constrained sub-CAs. For example, consider this certificate https://crt.sh/?id=21606064 https://crt.sh/?id=21606064 . It was issued by DigiCert to Microsoft, granting Microsoft the ability to provide OCSP responses for any certificate issued by Digicert's Baltimore CyberTrust Root. We know from DigiCert's disclosures that this is independently operated by Microsoft. So my understanding is this: The CA's have issued certificates/sub-CA certs without the proper extension (or with too many extensions), causing those to be able to sign a OCSP response. And the Online Certificate Status Protocol (OSCP) is used to check the revocation status of certificates with the CA. So, this would allow e.g Microsoft to generate a fake OCSP response? That would perhaps be useful in some kind of MITM-attack scenario? While not good, perhaps not an end of the world problem either? However, I wonder how much problem will come for people needing to replace those soon to be revoked sub-CA certs...

bluesign 6y ago
If intermediate certificate private key leaks, basically you can block revocation. As you can sign OCSP message with this certificate. Kinda defeats purpose of revocation. But as far as I know, browsers are not failing hard on OCSP failure, if you can mitm the connection possibly you can block OCSP requests too.
steventhedev 6y ago
Many of the violations discussed on the security lists are not end of the world at all. The 63-bit instead of 64-bit serial number entropy issue is a good example of this. But the strict enforcement of all violations makes it easier to spot bad actors or at least those who aren't competently handling all of the requirements to be a CA. Bottom line: the entire CA system is built on trust. Would you trust someone who doesn't take issues seriously because they think they're small or unimportant? EDIT: reading the full report, it seems that the underlying risk is that if one of the intermediate CAs were to be compromised, even if it was revoked it could theoretically forge an OCSP response that it is still valid (and as a trusted CA issue certs for anything). So the response is very appropriate given the potential impact.

Mojah 6y ago

Shameless plug, but I built a monitoring solution with extensive SSL checks that’ll catch & report these if they’re revoked: https://ohdear.app https://ohdear.app

agustif 6y ago
This looks nice, I've recently started using instatus.com for the status page, but love your monitoring parts, might use the trial to further check it out!

deleted 6y ago

[deleted]

JakeWesorick 6y ago

What's an easy way to test if I am affected by this?

floatingatoll 6y ago
Contact the customer support of whoever issued your SSL certificates and ask if you are affected by this. If you use any of the major auto-issue, auto-renew certificate platforms, then you do not need to take any action — either you aren’t affected or they’ll issue a new certificate to you automatically. (Let’s Encrypt, AWS Certificate Manager, Google-Managed SSL Certificates, Heroku Automated Certificate Management)
Mojah 6y ago
Hi Jake! Enter your website URL in here, and it'll tell you if you're affected or not: https://ohdear.app/tools/certificate https://ohdear.app/tools/certificate
tgsovlerkhgsel 6y ago
Check if your browser shows a lock symbol on any important sites you visit. (This affects the PKI as a whole, because a single unconstrained compromised sub-CA can misissue for any domain - so if you use HTTPS, you're affected). If your question is whether you should take action, the answer is no, unless you're in some way responsible for an intermediate CA, which is something you'd know about.

zenexer 6y ago

The link is very technical, resulting in some confusion as to why this is such a big problem. The comments on HN reflect that. Here’s my understanding: This isn’t a problem because a sub-CA can revoke any certificate from any other sub-CA of the same CA. That would be bad, but, at worst, it’s denial-of-service. Rather, this is a problem because any sub-CA can effectively reverse the revocation of any other sub-CA, or the CA itself. That’s immensely problematic. Suddenly, the CA has no reliable way fully revoke certificates. Revocation is already somewhat broken as it is, but this gives a lot of entities the ability to deliberately interfere with revocation in ways that they shouldn’t be able to. The author goes on to explain that revocation of the affected certificates is insufficient, because they could be used to effectively reverse their own revocation at any point in the future. Instead, it must be proven that all copies of the keys have been destroyed. That’s quite an undertaking. What the author fails to mention is that revocation is already pretty broken. Most major browsers have their own built-in CRL replacements that contain the most important revocations they need to know about. Some browsers, like Firefox, may make additional efforts to ensure that any given certificate hasn’t been revoked; others, like Chrome, don’t. If you’ve ever visited a site that gives you a certificate error in Firefox but not Chrome, that’s likely why. In the case of browsers, it should be possible for each browser to forcefully revoke affected certificates, but revoking a sub-CA certificate is quite disruptive, so I’d be surprised if that happens within 7 days. The catch is that this technique is really only effective in modern, up-to-date browsers. In any case, the title is misleading. I don’t see where the author guarantees that this will happen within 7 days. The author claims it should happen within 7 days, but considering that the damage is already done and cannot be fully reversed by revocation, I find it hard to believe enforcing that deadline makes sense here.

zenexer 6y ago
Flagging for a title change, as the revocation deadline was proposed by a single individual on a mailing list, and there’s no evidence from further discussion in that thread that a consensus has been reached. I would not expect all these certificates to be revoked within 7 days.
- mbrinkers 6y ago

yread 6y ago

Wow https://crt.sh/?id=7846179 https://crt.sh/?id=7846179 is also on the list, the certificate used by Dutch government to authenticate citizens. Yikes.

floatingatoll 6y ago

Digicert’s reply is called out by the original poster as being particularly high quality: https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg13535.html https://www.mail-archive.com/dev-security-policy@lists.mozil...

terom 6y ago

There's some related technical background in this 2014 bug report on Firefox rejecting a TLS server certificate issued by such an intermediate CA with the problematic 'OCSP Signing' EKU: https://bugzilla.mozilla.org/show_bug.cgi?id=991209#c7 https://bugzilla.mozilla.org/show_bug.cgi?id=991209#c7 The problem with intermediate CAs with the 'OCSP Signing' EKU being able to sign OCSP responses for any sibling certificates issued by the root CA seems to have been recognized for a while. But in this case, Mozilla allowed and ignored the 'OCSP Signing' EKU for vendor interop with existing intermediate CAs. Mozilla products would reject such OCSP responses, but the CA policy on issuing intermediate CA certificates with the 'OCSP Signing' EKU was not fixed. Specifically, MSADCS (Microsoft software used to issue certificates) requiring the CA certificate to have the 'OCSP Signing' EKU, "by design": https://support.microsoft.com/en-us/help/2962991/you-cannot-enroll-in-an-online-certificate-status-protocol-certificate https://support.microsoft.com/en-us/help/2962991/you-cannot-... The situation seems to be that the MSADCS policy on requiring the 'OCSP Signing' EKU on the intermediate CA certificate is incorrect, and clients validating OCSP responses signed by a delegated OCSP responder certificate do not require the 'OCSP Signing' EKUs on the issuing (intermediate/root) CA certificate(s): https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/XQd3rNF4yOo/bXYjt1mZAwAJ https://groups.google.com/forum/#!msg/mozilla.dev.security.p... This is in contrast to e.g. the TLS serverAuth EKU, where these EKUs chain: a technically constrained CA certificate must have a TLS serverAuth EKU in order to issue end-entity certificates with a TLS serverAuth EKU. The 'OCSP Signing' EKU on the intermediate CA certificate is unnecessary: the intermediate CA does not need the EKU to sign OCSP responses directly, nor to issue a delegated OCSP responder certificate with the 'OCSP Signing' EKU. The unnecessary 'OCSP Signing' EKU on the intermediate CA certificate is harmful, because it may be interpreted as a delegated OCSP responder certificate for the issuing root CA, capable of signing OCSP responses for any certificate issued by the root CA. The workaround for MSADCS is to use an untrusted CA certificate (e.g. self-signed) with the same public key and subject DN as the intermediate CA to issue the OCSP responder certificate, adding the 'OCSP Signing' EKU to both the OCSP responder certificate and the workaround-CA. The resulting OCSP responder certificate with the 'OCSP Signing' EKU will validate as an OCSP responder certificate for the actual intermediate CA, without including the 'OCSP Signing' EKU in the CA certificate. This BR policy violation approach seems to be an attempt to highlight the security problems and fix this long-standing issue, limiting the use of the 'OCSP Signing' EKU in order to protect clients from OCSP responses signed by intermediate CA certificates that were not intended to be delegated OCSP responder certificates for their root CAs.

dghughes 6y ago

Anyone remember two years ago when the Comodo CEO emailed hundred of private keys?

tialaramex 6y ago
That isn't actually what happened, but I don't doubt that's how you've remembered it. Comodo at this point controlled the CA roots that had belonged to Symantec. Trustico, a Symantec reseller (same sort of relationship to Symantec that your local Ford dealer has to the Ford motor company) asked Comodo to mass revoke thousands of certificates it had sold to third party subscribers as reseller for Symantec. It's not clear what Trustico hoped to achieve by that, maybe they believed they could get back the cost of the certificates? We don't know the details of the (confidential) contract between Trustico and Symantec or to what extent the contract terms survived transfer to Comodo. Maybe Trustico just wanted to push its customers into new deals, because it was not a Comodo reseller and risked being frozen out. Anyway, Jeremy Rowley, a Comodo VP asked for a reason to revoke these certificates, and by return he got thousands of private keys. Private Key compromise is a valid reason for revoking certificates, so Rowley confirmed the certificates matched these private keys and Comodo began revoking them. Trustico are the people who had thousands of private keys. A CA is strictly prohibited from having your private keys (and as we saw, if they are shown them they should revoke your certificates) but of course whether a CA enforces this rule on its resellers (via contract terms) is a matter between the CA and reseller. The whole point of private keys is that they're private. So, in one sense Trustico's customers got what they deserved - do not give your private keys to some reseller or trust them to pick keys for you. Nothing went wrong at Comodo here. And if you as a subscriber followed good practices you weren't affected either even if you'd bought certificates through Trustico. Only customers who'd gone with Trustico and done something inherently unsafe got burned.
- sleevi 6y ago
  You said Comodo throughout, but it was DigiCert :)
  - tialaramex 6y ago

tptacek 6y ago

I was going to blog this after tormenting Sleevi on Twitter with more dumb questions, but I guess we're just going to do this here. The issue† --- it's super complicated in its particulars but not in its outline --- is that CAs have been issuing constrained intermediate CAs (CA's that can sign only a subset of certificates) that, because of a misconfiguration, can sign any OCSP message for their root CA. So, for instance, Sleevi points out a HARICA (Greek university CA) cert that is constrained by dint of not having the serverAuth EKU (so it can't sign TLS certs), but because it has the OCSPSigning EKU, can be used to sign OCSP messages. What happened was this: lots of CAs run by companies in the wild (remember: there are all sorts of constrained CAs running inside companies that you aren't supposed to have to care about, in part because they're constrained) run on Microsoft's CA software. Many of those CAs want to support OCSP. The way you generally set up OCSP is to sign a Delegated Responder certificate, which is an end-entity (non-CA) cert with the OCSPSigning EKU, which says "this certificate can be used to sign OCSP certs for this part of the PKI". But Microsoft's CA is broken: it won't let you sign a cert with the OCSPSigning EKU unless your CA cert also has that EKU (the EKUs must "chain"). So, that's what CAs did, despite the fact that chaining EKUs like that changed the semantics of what the certs were intended to express. So, as Sleevi put it, basically a bunch of CAs accidentally signed the OCSP equivalent of a bunch of CA:TRUE certificates. They gave their customers the ability to essentially disable revocation for the whole CA. We can go back and forth on how sound the WebPKI revocation infrastructure is with or without this mistake. But at a minimum we should be able to stipulate that strengthening revocation is an important project for browser vendors, and CAs that accidentally break the revocation infrastructure by misissuing certificates are an impediment to that project. There's an interesting backstory to this, which is what I actually wanted to write about, and that's CA/B Forum SC31. The CA/B Forum is the standards body that coordinates between browsers and CAs; they maintain the BRs, which are the bylaws for operating a CA that browsers will trust. The politics of CA/B Forum are weird, because CAs and browsers are structurally adversarial: browsers want maximal security regardless of the commercial implications for CAs (as they should). As a result of that weirdness, the browser vendors rely not only on the BRs, but also on their own bylaws for their respective root certificate programs. SC31 is an attempt by Ryan Sleevi to align the CA-approved BRs with the browser-approved root programs. Of course, the browser root programs, particularly Mozilla and Google's, are the only thing that really matters, because those rules determine whether the browsers themselves will honor a CA's certificates. But the CA/B Forum is dominated by CAs who sort of dispute this fact. So, SC31 asks that the BRs import that (now common) browser root program rule that certs can only live for just over a year. The "one year only" rule was considered previously by the CA/B Forum and failed in a vote (CA customers don't like that rule), so the CA's are unhappy to be relitigating that point. But then, the litigation itself is kind of theatrical, in that browsers already made this decision and mooted the debate. Standards bodies are weird. Anyways, what makes this funny is that, while the one-year restriction is the real problem that seemed to piss off the CAs in SC31, they also challenged some of its OCSP language. In the course of debating with them about whether SC31 described reality or not, Sleevi started looking at their OCSP issuances. And here we are: the CAs aren't even following their own BRs, and are breaking their bylaws in ways that materially impact TLS security. I understood literally none of this when I first read Ryan's message. I thought I knew some stuff about the WebPKI, but even with help on Twitter I had trouble with this eldritch stuff. I don't know how people like Sleevi do it, and, apparently, neither do many CA operators. This probably would have been a boring blog post anyways, but to make Kurt happy, I'll conclude with: "use Fly.io!". :) † Wrinkle: Sleevi flags these CA certs as missing the ocsp-nocheck attribute, which is presumably how he spotted them. ocsp-nocheck says "this certificate can't be trusted to OCSP revoke itself, because that's silly"; the idea is, certs flagged ocsp-nocheck rely on very short lifetimes rather than OCSP to manage revocation. Very short lifetimes are things you expect on end-entity certificates, which is what OCSP Delegated Responders are, but not so much on CA:TRUE certificates, which are what Sleevi actually found.

Mojah 6y ago

Here's a very quick way to check if you may be affected: this will check all certificates in the chain to see if they match the fingerprint of the to-be-revoked list. https://ohdear.app/tools/certificate https://ohdear.app/tools/certificate