13 ms·

A New Life for Certificate Revocation Lists

121 points by grappler 4y ago

tinus_hn 4y ago
Is certificate revocation really so common a crl of a normal authority is gigabyte sized?
- schoen 4y ago
  No, not at all. The bad case would be if Let's Encrypt discovers a problem (like a security flaw or implementation error in a validation method, as happened with the TLS-ALPN-01 method before) and concludes that it has to mass-revoke a very large number of affected certificates.
  - hinkley 4y ago
    Or perhaps if another CloudBleed style incident ever happens.
- roblabla 4y ago
  The problem is, you need to design for the worse case scenario here, because when the worse case does happen, the last thing you need is for your revocation system to not work because it doesn't scale. So, no, it's not common. But it's necessary.
- er4hn 4y ago
  The DoD uses x509, and CRLs, in those Common Access Cards (CAC) everyone in the org has. Since this covers most of the armed forces, that's fairly large. As of 2012[1] this was around 200 MB of CRLs and was only expected to get larger over time. [1] https://dl.dod.cyber.mil/wp-content/uploads/pki-pke/pdf/unclass-certvalidation_capability_require-bestpractices.pdf https://dl.dod.cyber.mil/wp-content/uploads/pki-pke/pdf/uncl... - Pg 7, under Local Cache
  - OrvalWintermute 4y ago
    As mentioned above, DoD uses smart strategies around certificate validation. CDNs + Localized OCSP + Tactical OCSP + Smart OCSP Clients + Network caching + OCSP & CRLs on the filesystem, just to name a few (not including delta CRLs and other solutions). The DoD OCSP Responders are configured to share hash sets with downstream OCSP Responders & Repeaters, which makes promulgation particularly easy.
- michaelt 4y ago
  According to [1] "On a typical Monday, we would expect to see a total of around 22,000-30,000 SSL certificates being revoked over the course of the day." i.e. ~8 million per year. [3] meanwhile says "1.8 million certificates are revoked per year" Looking at a random CRL [2] it's 41 bytes per revoked certificate. 8 million records at 41 bytes per record would be 300+ Megabytes. And a cautious CA might keep revoked certificates in their CRL for more than a year. So if an event like heartbleed happened again and uncommonly large numbers of certificates needed to be revoked, the gigabyte range is within the bounds of possibility. [1] https://news.netcraft.com/archives/2014/04/11/heartbleed-certificate-revocation-tsunami-yet-to-arrive.html https://news.netcraft.com/archives/2014/04/11/heartbleed-cer... [2] http://crl3.digicert.com/Omniroot2025.crl http://crl3.digicert.com/Omniroot2025.crl [3] https://www.grc.com/revocation/crlsets.htm https://www.grc.com/revocation/crlsets.htm
  - WorldMaker 4y ago
    > And a cautious CA might keep revoked certificates in their CRL for more than a year. A CA does need to keep the revoked certificate in the CRL until at least the natural expiration date on the certificate, so for the CAs that give certificates out with 5 years out or more expiration dates, they may need to keep CRLs for much longer than just a year just naturally by nature of their expiration dates.
    - tialaramex 4y ago
      This CRL rule is about the Web PKI, for which currently policy requires new certificates expire after no more than 398 days. The previous status was the certificates could last up to 825 days, however that policy changed at the end of August 2020, so there are no extant certificates under those rules which expire after this year. And before that the policy was 39 months, but the last such certificate expired in 2021. Before that the policy was 5 years, but that policy changed in 2015 and so such certificates are long expired.
xoa 4y ago
I was kind of surprised that OCSP stapling didn't get any mention at all. I thought that was a major improvement in both resource cost and privacy? Since the time stamped response is proxied by the site operator rather then going directly to the CA, the load is almost entirely switched to the site itself, and the site itself is the only one who knows a given IP is asking for it, which is fine because obviously the site knows a given browser is connecting to it anyway. It's decentralized again. I vaguely remember at one point there was a major limitation of only supporting a single OCSP response at a time but I thought that was dealt with via a later RFC and then entirely obviated as an issue by TLS 1.3. Did something happen there or some other significant issue get discovered? I'm curious why the move back to CRLs (albeit improved) vs must-staple. It seemed like a reasonable elegant and straight forward solution that fit the web pretty well.
- champtar 4y ago
  I also love OCSP stapling but there are some limitations: - webserver need to implement it - admin need to enable it - webserver need internet access Maybe they saw with some telemetry that very few website actually enable OCSP stampling and decided to implement a fix that cover all certs and can really be deployed
  - coffee-- 4y ago
    Firefox Telemetry shows Beta 104 users encountered stapling on 13.95% of TLS handshakes. [1] The stapling telemetry is no longer turned on in Release [2], and even if it were, you have to do special things to look at Release data, but some years back (~2018 maybe?) I remember Release stapling was substantially lower than the more tech-savvy Beta and Nightly populations. Which is pretty normal, as tech-oriented sites are more likely to turn on advanced features. [1] https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2022-08-18&include_spill=0&keys=__none__!__none__!__none__&max_channel_version=beta%252F104&measure=SSL_OCSP_STAPLING&min_channel_version=nightly%252F55&processType=*&product=Firefox&sanitize=1&sort_by_value=0&sort_keys=submissions&start_date=2022-07-25&table=0&trim=1&use_submission_date=0 https://telemetry.mozilla.org/new-pipeline/dist.html#!cumula... [2] "prerelease" https://probes.telemetry.mozilla.org/?search=stapl&view=detail&probeId=histogram%2FSSL_OCSP_STAPLING https://probes.telemetry.mozilla.org/?search=stapl&view=deta...
ok_dad 4y ago
Browsers and CAs still deciding for the user what's best with yet another centralized database that we have to "trust" is complete and implemented correctly. I don't see how this is so hard: just let us download the CRLs. Maybe add them on a torrent-like system so they can be shared (and validated) peer-to-peer, or at least have hundreds or thousands of mirrors that can provide that data. All this "it's too big and the user would have to download it" bullshit is just pushing us more and more into "computing as a service"; aka: we OWN your digital life. Edit: I was partly wrong, this is a good thing because you CAN download the CRLs now (see comments below here for info), whereas previously you couldn't. Your browser still probably won't support a full CRL download, but I could be pleasantly surprised.
- agwa 4y ago
  > I don't see how this is so hard: just let us download the CRLs Previously, you couldn't do this, because not all CAs published CRLs. Beginning October 1, you will be able to just download the CRLs, because Apple and Mozilla are requiring it. It's therefore unclear what your beef is,
  - ok_dad 4y ago
    > Beginning October 1, you will be able to just download the CRLs Correction: Apple and Mozilla will be able to just download the CRLs. Not me. The link in the post SPECIFICALLY says us common plebes don't get that right.
    - agwa 4y ago
      Where does the post say that? If you think it's because the URLs will be disclosed in the CCADB, note that the contents of the CCADB are published here: https://www.ccadb.org/resources https://www.ccadb.org/resources Specifically, the CRL URLs can be found in this CSV file: http://ccadb-public.secure.force.com/ccadb/AllCertificateRecordsCSVFormat http://ccadb-public.secure.force.com/ccadb/AllCertificateRec...
      - 4y ago
luhn 4y ago
> If we had an incident where we needed to revoke every single one of those certificates at the same time, the resulting CRL would be over 8 gigabytes. I don't know much about this stuff, so apologies if this is a silly question: If you needed to revoke all the certificates, couldn't you just revoke the handful of intermediary certificates and call it a day? I assume you'd want to revoke them anyways if there's a situation severe enough that merits revoking 200 million certificates.
- mholt 4y ago
  It's a good question, if I read you rightly. There is no "handful" of intermediate certificates -- there are precisely 4 (for Let's Encrypt [0]) and they are essentially on-line root certificates. And if those certificates aren't even compromised, revoking them would only harm the ecosystem. [0]: https://letsencrypt.org/certificates/ https://letsencrypt.org/certificates/
  - shallichange 4y ago
    According to the link you posted, there are intermediate CAs. Those could be revoked and effectively revoke all the end entity certificates.
    - tialaramex 4y ago
      When they write "essentially on-line root certificates" they don't mean that they're literally on-line root certificates, because that's prohibited. They're essentially on-line root certificates because they serve most of the function that such roots would serve if they were allowed. There aren't a bunch more available to replace them, so this means recovery now requires a key ceremony, figure on a week to a month to arrange that. Whereas if you're able to "just" revoke 10 million end entity certificates you can recover immediately.
- mcpherrinm 4y ago
  Yes, it is likely we’d revoke an intermediate if a significant fraction of all issued certs had to be revoked. But we do want our revocation infrastructure to support revoking all certs if needed. We have a set of backup intermediates that can be activated if we had to revoke the active ones for any reason, so the disruption wouldn’t be too high hopefully. (I work at Let’s Encrypt, but this is my own opinion and not that of my employer)
frankjr 4y ago
> This means that they’re often very large – easily the size of a whole movie. Couldn't they just use actual units? This says absolutely nothing.
- TheSpiceIsLife 4y ago
  One Olympic swimming football bus tree worth of data. When I was reading the thread the other day about a trees worth of oxygen from the MOXIE experiment, I couldn't help thinking: why not just use a term everyone is familiar with, litres per minute air. Enough oxygen to sustain an adult at rest for x minutes. I'm beginning to suspect there's an in-joke with science / tech writers about strained analogies. And I'm not in.
  - JohnFen 4y ago
    > I'm beginning to suspect there's an in-joke with science / tech writers about strained analogies. Which has been going on for longer than I've been alive. The idea (I assume) is to take a large number that's hard to conceive and turn it into something everyone can relate to. But inevitably, they choose things that few can actually relate to, or things that are so vague/variable to be meaningless. It just adds more confusion all around. It has to be an intentional joke.
- WorldMaker 4y ago
  The article does later state Lets Encrypt's own expected worst case is 8 GBs in one CRL file in the hopefully unlikely scenario that every unexpired certificate they manage was revoked.
  - OrvalWintermute 4y ago
    Merkle Hash Trees are the well-known solution for this, whenever they decide to update the protocols (was software patent encumbered til 2017)
    - WorldMaker 4y ago
phlip9 4y ago
I can't wait for some big site certs to false positive in a CRL bloom filter and cause a big outage : )
- coffee-- 4y ago
  CRLite builds a cascade of Bloom filters to ensure no false positives. For Firefox end users, a certificate only gets tested against the filter cascade if it is known to have been included in its creation (by examining the embedded SCT timestamps). If it's not definite that the certificate was used to generate the filter, then Firefox reverts to OCSP. (I'm one of the authors of CRLite in Firefox: https://insufficient.coffee/2020/12/01/crlite-part-4-infrastructure-design/ https://insufficient.coffee/2020/12/01/crlite-part-4-infrast... )
  - phlip9 4y ago
    Ahh thanks for the link and sorry for the snark; the finite universe optimization is cool! [0] # Why is CRLite able to compress so much data? Bloom filters are probabilistic data structures with an error rate due to data collisions. However, if you know the whole range of data that might be tested against the filter, you can compute all the false positives and build another layer to resolve those. Then you keep going until there are no more false positives. In practice, this happens in 25 to 30 layers, which results in substantial compression. EDIT: Is there any risk of filter blow up (think 1000's of layers) if a CA did a mass revocation (maybe some root key leak)? [0] https://github.com/mozilla/crlite/wiki#why-is-crlite-able-to-compress-so-much-data https://github.com/mozilla/crlite/wiki#why-is-crlite-able-to...
    - coffee-- 4y ago
      Mozilla worked with some researchers to analyze a bunch of degenerate cases like that and basically the answer is 'no', that the CRLite paper's calculation of an optimal false positive rate per layer works out quite well. What does happen in CRLite is that you can't keep shipping the tiny "stash" updates to clients, you have to mint a whole new .mlbf filter file, which is about a megabyte. [Edit:] Then you can resume the "stash" updates from there, but the ecosystem 'shock' requires a regeneration of the filter. (There was supposed to be a blogpost on the Mozilla blog from the research teams; I don't know if it was ever written.)
OrvalWintermute 4y ago
Lots of this information is completely bogus. > But because OCSP infrastructure has to be running constantly and can suffer downtime just like any other web service, most browsers treat getting no response at all as equivalent to getting a “not revoked” response. This means that attackers can prevent you from discovering that a certificate has been revoked simply by blocking all of your requests for OCSP information. This is false. Non-nonce OCSP is inherently cachable, and replayable. That means you can have your own HA setups with HA OCSP clients talking to HA OCSP servers (repeaters & responders) backed up by caching in commercial CDNs, and local caching servers like bluecoats. Likewise, OCSP stapling helps remove much of the performance and privacy issues, pushing it to the serving webserver. Beyond this, you can just use squid or localized HA OCSP services, and do some DNS rewriting to support it even more HA. Nonced OCSP is the rare beast that needs to be online, but there are HA OCSP with smart OCSP clients. > To help reduce load on a CA’s OCSP services, OCSP responses are valid and can be cached for about a week. But this means that clients don’t retrieve updates very frequently, and often continue to trust certificates for a week after they’re revoked. Trust Stores are inherently manageable. The lag around revocation completely depends on CRL/OCSP publishing, and client update requests. > And perhaps worst of all: because your browser makes an OCSP request for every website you visit, a malicious (or legally compelled) CA could track your browsing behavior by keeping track of what sites you request OCSP for. This is why we advocate OCSP Stapling and use of CDNs for OCSP & CRL cache hits. Furthermore, localized OCSP mentioned above decentralizes this even further. > So both of the existing solutions don’t really work: CRLs are so inefficient that most browsers don’t check them, and OCSP is so unreliable that most browsers don’t check it. We need something better. CRLs & OCSP work pretty well when actually supported. When Diginotar happened I polled every single publicly available commercial CA - strangely, a ton of them were not producing any CRL/OCSP at all, putting clients into a fail-open mode. Lesson of the story: don't blame a protocol for lazy CAs, bad implementations, or the lack of operational excellence from many vendors.
jnwatson 4y ago
What about if you’re calling an API? Is there compressed CRL support in libcurl or similar?
cryptonector 4y ago
> There’s still a long way to go before revocation in the Web PKI is truly fixed. It won't be fixed until we have name constraints on CA certificates, and a way to decorate trust anchors with local policy name constraints.
jbverschoor 4y ago
Shitty title for an article considering recent events
deleted 4y ago
[deleted]
weinzierl 4y ago
> "They process the CRLs into a smaller format such as a Bloom filter, then push the new compressed object to all of the installed browser instances using pre-existing rapid update mechanisms. Firefox, for example, is pushing updates as quickly as every 6 hours." Does anyone know what the rate of revocations is? I can easily imagine a situation where it is high enough to cause a browser update every few hours. That is for every installed browser, because - as they say in the article - Browser-Summarized CRLs are "proprietary, browser-specific CRLs". Moreover there are non-browser clients which we must consider if we are to take this proposal seriously. ------ My quick back of the envelope estimation: CRL size: 4GiB (they say in the article that it could be easily the size of a movie) Average Cert Size: 75 bytes (first hit in Google, no idea if reliable number) Time Span: 825 days (CRLs have only unexpired certs and ones older than 825 should all be expired) 4GiB/(75B/cert)/825days = 69141 certs/day Sounds way too high to me. Where am I wrong?
- lifthrasiir 4y ago
  In the CRLite paper [1] the bandwidth cost was estimated to be ~600 KB per day. Most crucially they do not directly store certificates but only boolean flags for hashes of virtually all once-valid certificates in existence. This is made possible because of Certificate Transparency; once the signature has been verified there can be only so many hashes to check. [1] https://obj.umiacs.umd.edu/papers_for_stories/crlite_oakland17.pdf#page=2 https://obj.umiacs.umd.edu/papers_for_stories/crlite_oakland...

121 points by grappler 4y ago

tinus_hn 4y ago

Is certificate revocation really so common a crl of a normal authority is gigabyte sized?

schoen 4y ago
No, not at all. The bad case would be if Let's Encrypt discovers a problem (like a security flaw or implementation error in a validation method, as happened with the TLS-ALPN-01 method before) and concludes that it has to mass-revoke a very large number of affected certificates.
- hinkley 4y ago
  Or perhaps if another CloudBleed style incident ever happens.
roblabla 4y ago
The problem is, you need to design for the worse case scenario here, because when the worse case does happen, the last thing you need is for your revocation system to not work because it doesn't scale. So, no, it's not common. But it's necessary.
er4hn 4y ago
The DoD uses x509, and CRLs, in those Common Access Cards (CAC) everyone in the org has. Since this covers most of the armed forces, that's fairly large. As of 2012[1] this was around 200 MB of CRLs and was only expected to get larger over time. [1] https://dl.dod.cyber.mil/wp-content/uploads/pki-pke/pdf/unclass-certvalidation_capability_require-bestpractices.pdf https://dl.dod.cyber.mil/wp-content/uploads/pki-pke/pdf/uncl... - Pg 7, under Local Cache
- OrvalWintermute 4y ago
  As mentioned above, DoD uses smart strategies around certificate validation. CDNs + Localized OCSP + Tactical OCSP + Smart OCSP Clients + Network caching + OCSP & CRLs on the filesystem, just to name a few (not including delta CRLs and other solutions). The DoD OCSP Responders are configured to share hash sets with downstream OCSP Responders & Repeaters, which makes promulgation particularly easy.
michaelt 4y ago
According to [1] "On a typical Monday, we would expect to see a total of around 22,000-30,000 SSL certificates being revoked over the course of the day." i.e. ~8 million per year. [3] meanwhile says "1.8 million certificates are revoked per year" Looking at a random CRL [2] it's 41 bytes per revoked certificate. 8 million records at 41 bytes per record would be 300+ Megabytes. And a cautious CA might keep revoked certificates in their CRL for more than a year. So if an event like heartbleed happened again and uncommonly large numbers of certificates needed to be revoked, the gigabyte range is within the bounds of possibility. [1] https://news.netcraft.com/archives/2014/04/11/heartbleed-certificate-revocation-tsunami-yet-to-arrive.html https://news.netcraft.com/archives/2014/04/11/heartbleed-cer... [2] http://crl3.digicert.com/Omniroot2025.crl http://crl3.digicert.com/Omniroot2025.crl [3] https://www.grc.com/revocation/crlsets.htm https://www.grc.com/revocation/crlsets.htm
- WorldMaker 4y ago
  > And a cautious CA might keep revoked certificates in their CRL for more than a year. A CA does need to keep the revoked certificate in the CRL until at least the natural expiration date on the certificate, so for the CAs that give certificates out with 5 years out or more expiration dates, they may need to keep CRLs for much longer than just a year just naturally by nature of their expiration dates.
  - tialaramex 4y ago
    This CRL rule is about the Web PKI, for which currently policy requires new certificates expire after no more than 398 days. The previous status was the certificates could last up to 825 days, however that policy changed at the end of August 2020, so there are no extant certificates under those rules which expire after this year. And before that the policy was 39 months, but the last such certificate expired in 2021. Before that the policy was 5 years, but that policy changed in 2015 and so such certificates are long expired.

xoa 4y ago

I was kind of surprised that OCSP stapling didn't get any mention at all. I thought that was a major improvement in both resource cost and privacy? Since the time stamped response is proxied by the site operator rather then going directly to the CA, the load is almost entirely switched to the site itself, and the site itself is the only one who knows a given IP is asking for it, which is fine because obviously the site knows a given browser is connecting to it anyway. It's decentralized again. I vaguely remember at one point there was a major limitation of only supporting a single OCSP response at a time but I thought that was dealt with via a later RFC and then entirely obviated as an issue by TLS 1.3. Did something happen there or some other significant issue get discovered? I'm curious why the move back to CRLs (albeit improved) vs must-staple. It seemed like a reasonable elegant and straight forward solution that fit the web pretty well.

champtar 4y ago
I also love OCSP stapling but there are some limitations: - webserver need to implement it - admin need to enable it - webserver need internet access Maybe they saw with some telemetry that very few website actually enable OCSP stampling and decided to implement a fix that cover all certs and can really be deployed
- coffee-- 4y ago
  Firefox Telemetry shows Beta 104 users encountered stapling on 13.95% of TLS handshakes. [1] The stapling telemetry is no longer turned on in Release [2], and even if it were, you have to do special things to look at Release data, but some years back (~2018 maybe?) I remember Release stapling was substantially lower than the more tech-savvy Beta and Nightly populations. Which is pretty normal, as tech-oriented sites are more likely to turn on advanced features. [1] https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2022-08-18&include_spill=0&keys=__none__!__none__!__none__&max_channel_version=beta%252F104&measure=SSL_OCSP_STAPLING&min_channel_version=nightly%252F55&processType=*&product=Firefox&sanitize=1&sort_by_value=0&sort_keys=submissions&start_date=2022-07-25&table=0&trim=1&use_submission_date=0 https://telemetry.mozilla.org/new-pipeline/dist.html#!cumula... [2] "prerelease" https://probes.telemetry.mozilla.org/?search=stapl&view=detail&probeId=histogram%2FSSL_OCSP_STAPLING https://probes.telemetry.mozilla.org/?search=stapl&view=deta...

ok_dad 4y ago

Browsers and CAs still deciding for the user what's best with yet another centralized database that we have to "trust" is complete and implemented correctly. I don't see how this is so hard: just let us download the CRLs. Maybe add them on a torrent-like system so they can be shared (and validated) peer-to-peer, or at least have hundreds or thousands of mirrors that can provide that data. All this "it's too big and the user would have to download it" bullshit is just pushing us more and more into "computing as a service"; aka: we OWN your digital life. Edit: I was partly wrong, this is a good thing because you CAN download the CRLs now (see comments below here for info), whereas previously you couldn't. Your browser still probably won't support a full CRL download, but I could be pleasantly surprised.

agwa 4y ago
> I don't see how this is so hard: just let us download the CRLs Previously, you couldn't do this, because not all CAs published CRLs. Beginning October 1, you will be able to just download the CRLs, because Apple and Mozilla are requiring it. It's therefore unclear what your beef is,
- ok_dad 4y ago
  > Beginning October 1, you will be able to just download the CRLs Correction: Apple and Mozilla will be able to just download the CRLs. Not me. The link in the post SPECIFICALLY says us common plebes don't get that right.
  - agwa 4y ago
    Where does the post say that? If you think it's because the URLs will be disclosed in the CCADB, note that the contents of the CCADB are published here: https://www.ccadb.org/resources https://www.ccadb.org/resources Specifically, the CRL URLs can be found in this CSV file: http://ccadb-public.secure.force.com/ccadb/AllCertificateRecordsCSVFormat http://ccadb-public.secure.force.com/ccadb/AllCertificateRec...
    - 4y ago

luhn 4y ago

> If we had an incident where we needed to revoke every single one of those certificates at the same time, the resulting CRL would be over 8 gigabytes. I don't know much about this stuff, so apologies if this is a silly question: If you needed to revoke all the certificates, couldn't you just revoke the handful of intermediary certificates and call it a day? I assume you'd want to revoke them anyways if there's a situation severe enough that merits revoking 200 million certificates.

mholt 4y ago
It's a good question, if I read you rightly. There is no "handful" of intermediate certificates -- there are precisely 4 (for Let's Encrypt [0]) and they are essentially on-line root certificates. And if those certificates aren't even compromised, revoking them would only harm the ecosystem. [0]: https://letsencrypt.org/certificates/ https://letsencrypt.org/certificates/
- shallichange 4y ago
  According to the link you posted, there are intermediate CAs. Those could be revoked and effectively revoke all the end entity certificates.
  - tialaramex 4y ago
    When they write "essentially on-line root certificates" they don't mean that they're literally on-line root certificates, because that's prohibited. They're essentially on-line root certificates because they serve most of the function that such roots would serve if they were allowed. There aren't a bunch more available to replace them, so this means recovery now requires a key ceremony, figure on a week to a month to arrange that. Whereas if you're able to "just" revoke 10 million end entity certificates you can recover immediately.
mcpherrinm 4y ago
Yes, it is likely we’d revoke an intermediate if a significant fraction of all issued certs had to be revoked. But we do want our revocation infrastructure to support revoking all certs if needed. We have a set of backup intermediates that can be activated if we had to revoke the active ones for any reason, so the disruption wouldn’t be too high hopefully. (I work at Let’s Encrypt, but this is my own opinion and not that of my employer)

frankjr 4y ago

> This means that they’re often very large – easily the size of a whole movie. Couldn't they just use actual units? This says absolutely nothing.

TheSpiceIsLife 4y ago
One Olympic swimming football bus tree worth of data. When I was reading the thread the other day about a trees worth of oxygen from the MOXIE experiment, I couldn't help thinking: why not just use a term everyone is familiar with, litres per minute air. Enough oxygen to sustain an adult at rest for x minutes. I'm beginning to suspect there's an in-joke with science / tech writers about strained analogies. And I'm not in.
- JohnFen 4y ago
  > I'm beginning to suspect there's an in-joke with science / tech writers about strained analogies. Which has been going on for longer than I've been alive. The idea (I assume) is to take a large number that's hard to conceive and turn it into something everyone can relate to. But inevitably, they choose things that few can actually relate to, or things that are so vague/variable to be meaningless. It just adds more confusion all around. It has to be an intentional joke.
WorldMaker 4y ago
The article does later state Lets Encrypt's own expected worst case is 8 GBs in one CRL file in the hopefully unlikely scenario that every unexpired certificate they manage was revoked.
- OrvalWintermute 4y ago
  Merkle Hash Trees are the well-known solution for this, whenever they decide to update the protocols (was software patent encumbered til 2017)
  - WorldMaker 4y ago

phlip9 4y ago

I can't wait for some big site certs to false positive in a CRL bloom filter and cause a big outage : )

coffee-- 4y ago
CRLite builds a cascade of Bloom filters to ensure no false positives. For Firefox end users, a certificate only gets tested against the filter cascade if it is known to have been included in its creation (by examining the embedded SCT timestamps). If it's not definite that the certificate was used to generate the filter, then Firefox reverts to OCSP. (I'm one of the authors of CRLite in Firefox: https://insufficient.coffee/2020/12/01/crlite-part-4-infrastructure-design/ https://insufficient.coffee/2020/12/01/crlite-part-4-infrast... )
- phlip9 4y ago
  Ahh thanks for the link and sorry for the snark; the finite universe optimization is cool! [0] # Why is CRLite able to compress so much data? Bloom filters are probabilistic data structures with an error rate due to data collisions. However, if you know the whole range of data that might be tested against the filter, you can compute all the false positives and build another layer to resolve those. Then you keep going until there are no more false positives. In practice, this happens in 25 to 30 layers, which results in substantial compression. EDIT: Is there any risk of filter blow up (think 1000's of layers) if a CA did a mass revocation (maybe some root key leak)? [0] https://github.com/mozilla/crlite/wiki#why-is-crlite-able-to-compress-so-much-data https://github.com/mozilla/crlite/wiki#why-is-crlite-able-to...
  - coffee-- 4y ago
    Mozilla worked with some researchers to analyze a bunch of degenerate cases like that and basically the answer is 'no', that the CRLite paper's calculation of an optimal false positive rate per layer works out quite well. What does happen in CRLite is that you can't keep shipping the tiny "stash" updates to clients, you have to mint a whole new .mlbf filter file, which is about a megabyte. [Edit:] Then you can resume the "stash" updates from there, but the ecosystem 'shock' requires a regeneration of the filter. (There was supposed to be a blogpost on the Mozilla blog from the research teams; I don't know if it was ever written.)

OrvalWintermute 4y ago

Lots of this information is completely bogus. > But because OCSP infrastructure has to be running constantly and can suffer downtime just like any other web service, most browsers treat getting no response at all as equivalent to getting a “not revoked” response. This means that attackers can prevent you from discovering that a certificate has been revoked simply by blocking all of your requests for OCSP information. This is false. Non-nonce OCSP is inherently cachable, and replayable. That means you can have your own HA setups with HA OCSP clients talking to HA OCSP servers (repeaters & responders) backed up by caching in commercial CDNs, and local caching servers like bluecoats. Likewise, OCSP stapling helps remove much of the performance and privacy issues, pushing it to the serving webserver. Beyond this, you can just use squid or localized HA OCSP services, and do some DNS rewriting to support it even more HA. Nonced OCSP is the rare beast that needs to be online, but there are HA OCSP with smart OCSP clients. > To help reduce load on a CA’s OCSP services, OCSP responses are valid and can be cached for about a week. But this means that clients don’t retrieve updates very frequently, and often continue to trust certificates for a week after they’re revoked. Trust Stores are inherently manageable. The lag around revocation completely depends on CRL/OCSP publishing, and client update requests. > And perhaps worst of all: because your browser makes an OCSP request for every website you visit, a malicious (or legally compelled) CA could track your browsing behavior by keeping track of what sites you request OCSP for. This is why we advocate OCSP Stapling and use of CDNs for OCSP & CRL cache hits. Furthermore, localized OCSP mentioned above decentralizes this even further. > So both of the existing solutions don’t really work: CRLs are so inefficient that most browsers don’t check them, and OCSP is so unreliable that most browsers don’t check it. We need something better. CRLs & OCSP work pretty well when actually supported. When Diginotar happened I polled every single publicly available commercial CA - strangely, a ton of them were not producing any CRL/OCSP at all, putting clients into a fail-open mode. Lesson of the story: don't blame a protocol for lazy CAs, bad implementations, or the lack of operational excellence from many vendors.

jnwatson 4y ago

What about if you’re calling an API? Is there compressed CRL support in libcurl or similar?

cryptonector 4y ago

> There’s still a long way to go before revocation in the Web PKI is truly fixed. It won't be fixed until we have name constraints on CA certificates, and a way to decorate trust anchors with local policy name constraints.

jbverschoor 4y ago

Shitty title for an article considering recent events

deleted 4y ago

[deleted]

weinzierl 4y ago

> "They process the CRLs into a smaller format such as a Bloom filter, then push the new compressed object to all of the installed browser instances using pre-existing rapid update mechanisms. Firefox, for example, is pushing updates as quickly as every 6 hours." Does anyone know what the rate of revocations is? I can easily imagine a situation where it is high enough to cause a browser update every few hours. That is for every installed browser, because - as they say in the article - Browser-Summarized CRLs are "proprietary, browser-specific CRLs". Moreover there are non-browser clients which we must consider if we are to take this proposal seriously. ------ My quick back of the envelope estimation: CRL size: 4GiB (they say in the article that it could be easily the size of a movie) Average Cert Size: 75 bytes (first hit in Google, no idea if reliable number) Time Span: 825 days (CRLs have only unexpired certs and ones older than 825 should all be expired) 4GiB/(75B/cert)/825days = 69141 certs/day Sounds way too high to me. Where am I wrong?

lifthrasiir 4y ago
In the CRLite paper [1] the bandwidth cost was estimated to be ~600 KB per day. Most crucially they do not directly store certificates but only boolean flags for hashes of virtually all once-valid certificates in existence. This is made possible because of Certificate Transparency; once the signature has been verified there can be only so many hashes to check. [1] https://obj.umiacs.umd.edu/papers_for_stories/crlite_oakland17.pdf#page=2 https://obj.umiacs.umd.edu/papers_for_stories/crlite_oakland...