4 ms·

It looks like Let's Encrypt has come up with the best plan possible for the goal of balancing mitigation of the security risks with compatibility for users. If

It looks like Let's Encrypt has come up with the best plan possible for the goal of balancing mitigation of the security risks with compatibility for users. If their plan works out as they've laid out, and as certain recent commits in the boulder source code would suggest... It would seem they intend to: 1. Allow renewal of already-issued certificates to same account holder to revalidate via TLS-SNI-01 for some limited time period. 2. Whitelist certain shared infrastructure providers who have lots of LE certs in force and who have demonstrated that they are not vulnerable. I'm betting especially for those that manage the whole certificate retrieval and deployment process. It's not clear if this is temporary but longer than #1 or "temporary" but very long term or "temporary" until we come up with another inband process. 3. Otherwise TLS-SNI-01 is gone. Meanwhile, they'll work with the ACME WG to see if they can't figure out a better TLS-SNI method which would not be vulnerable. That looks less and less workable absent some special TLS extensions or ALPN and server support for those. For those who have options, it's worth pointing out that the purely DNS based validation methods are literally closest to the facts that the CA's validation process wishes to prove. Domain Control validated means that you're showing effective control of the domain. Nothing says I control the domain like the ability to add and remove data from the authoritative DNS servers for the domain label (and children thereof) in question.

bproven 9y ago
Agreed - looks like a good plan. It definitely must have been a busy few days to a week for them : First Meltdown/Spectre to consider and now this to deal with - however I am glad to see the update and the total transparency each step of the way!
BillinghamJ 9y ago
I think a similar process, but built off of ALPN rather than SNI, seems like the clear solution. Nobody at LE seems to have spoken about this or acknowledged it as a possible option yet though.
- mdhardeman 9y ago
  Ryan Sleevi of Google has discussed (specifically) such a possibility on the m.d.s.p group. Today's update from Let's Encrypt did allude under "ACME Protocol Updates" that there might be further work done to the protocol to attempt to remediate the risks. Probably they don't want to get specific because even if a concrete proposal were ready to begin coding today, it would take time to build the reference client, time to build server infrastructure, test, etc. Then before that work would have any benefit to the various websites needing validation, it would require server software upgrades to facilitate those extensions or ALPN negotiations. My utter speculation is that they're thinking it would likely take long enough that everyone will have to be off TLS-SNI-01 before its replacement becomes available.
  - BillinghamJ 9y ago
    Ah thank you - I have found that now. A link to that discussion, for reference: https://groups.google.com/d/msg/mozilla.dev.security.policy/RHsIInIjJA0/iXyaHh2DAQAJ https://groups.google.com/d/msg/mozilla.dev.security.policy/...
  - Titanous 9y ago
    Work has already started on a new challenge using ALPN: https://mailarchive.ietf.org/arch/msg/acme/mrKOeRK1K6H_42Hxbt3A1XtLFJM https://mailarchive.ietf.org/arch/msg/acme/mrKOeRK1K6H_42Hxb...
    - mdhardeman 9y ago
      Let's Encrypt has also just added a new post in which they've been working tirelessly on a new nginx and apache plugin to certbot utilizing HTTP-01 validation: https://community.letsencrypt.org/t/help-test-certbot-apache-and-nginx-fixes-for-tls-sni-01-outage/50207 https://community.letsencrypt.org/t/help-test-certbot-apache... It seems they are predicting TLS-SNI-0x going away for a lengthy period of time. That said, the ALPN proposal is a start. Though rather than just having it as a mere marker, it should incorporate features to securely indicate which domain label it is attempting to validate and achieve consensus on part of validator and the endpoint being validated. I am hopeful such a scheme may be useful for future deployments down the road. I think it is likely before there is infrastructure in place utilizing a new mechanism of that kind that current needs will need to be met with one of the other mechanisms. The speed and resource with which Let's Encrypt is working on solutions to migrate users to non-TLS-SNI validations might well be a signal.
will_hughes 9y ago
> Nothing says I control the domain like the ability to add and remove data from the authoritative DNS servers for the domain label (and children thereof) in question. Unfortunately this makes life difficult for us. We run a whitelabelled platform with tens of thousands of users - it's hard enough to get many people to understand setting a CNAME record (vs just setting an A record). Requiring them to update a TXT record once would be a big enough challenge, but doing it every 30-90 days is never going to happen. The smaller customers would probably be happy with us hosting their DNS, but we're not going to do that - the larger ones wouldn't. TLS-SNI being gone and DNS being unworkable means we're left with HTTP only.
- mdhardeman 9y ago
  But if you're a web post with 10k+ users, what's the problem with the HTTP-01 challenge? You just allow .well-known/* to be passed on to reflect the challenge responses you've generated for the client, while 301 redirecting everything else to their https:// https:// site. I'm confused how that would be harder for a web host at that scale? EDIT: I get people trying to run a server off their cable modem / rtr public IP, and 80 might be taken by something other than the target the port forward for 443 is going to -- and that's a problem for those use cases -- but that kind of concern wouldn't exist in a significant hosting infrastructure.
  - will_hughes 9y ago
    > what's the problem with the HTTP-01 challenge? Nothing, yet. But who's to say that another similar bug won't be found in common shared-hosting platforms that forces LE to turn that challenge off too?
    - mdhardeman 9y ago
      True. Because almost all of the CAs utilize a web control mechanism, with many of them probably having processes not as rigorous as HTTP-01, it is likely that there would be significant backlash and a lengthier migration away from the method for that case. That said, anyone who can would be well advised to figure out how their DNS based mechanism would work if it were ever needed. As I and others have pointed out, there are clever and fully supported hacks for validating dns-01 without dynamic control of the full domain zone. (CNAME to another zone for the _acme-challenge labels, NS delegation to refer each _acme-challenge label as an independent zone at a different NS, etc.)

bproven 9y ago

Agreed - looks like a good plan. It definitely must have been a busy few days to a week for them : First Meltdown/Spectre to consider and now this to deal with - however I am glad to see the update and the total transparency each step of the way!

BillinghamJ 9y ago

I think a similar process, but built off of ALPN rather than SNI, seems like the clear solution. Nobody at LE seems to have spoken about this or acknowledged it as a possible option yet though.

mdhardeman 9y ago
Ryan Sleevi of Google has discussed (specifically) such a possibility on the m.d.s.p group. Today's update from Let's Encrypt did allude under "ACME Protocol Updates" that there might be further work done to the protocol to attempt to remediate the risks. Probably they don't want to get specific because even if a concrete proposal were ready to begin coding today, it would take time to build the reference client, time to build server infrastructure, test, etc. Then before that work would have any benefit to the various websites needing validation, it would require server software upgrades to facilitate those extensions or ALPN negotiations. My utter speculation is that they're thinking it would likely take long enough that everyone will have to be off TLS-SNI-01 before its replacement becomes available.
- BillinghamJ 9y ago
  Ah thank you - I have found that now. A link to that discussion, for reference: https://groups.google.com/d/msg/mozilla.dev.security.policy/RHsIInIjJA0/iXyaHh2DAQAJ https://groups.google.com/d/msg/mozilla.dev.security.policy/...
- Titanous 9y ago
  Work has already started on a new challenge using ALPN: https://mailarchive.ietf.org/arch/msg/acme/mrKOeRK1K6H_42Hxbt3A1XtLFJM https://mailarchive.ietf.org/arch/msg/acme/mrKOeRK1K6H_42Hxb...
  - mdhardeman 9y ago
    Let's Encrypt has also just added a new post in which they've been working tirelessly on a new nginx and apache plugin to certbot utilizing HTTP-01 validation: https://community.letsencrypt.org/t/help-test-certbot-apache-and-nginx-fixes-for-tls-sni-01-outage/50207 https://community.letsencrypt.org/t/help-test-certbot-apache... It seems they are predicting TLS-SNI-0x going away for a lengthy period of time. That said, the ALPN proposal is a start. Though rather than just having it as a mere marker, it should incorporate features to securely indicate which domain label it is attempting to validate and achieve consensus on part of validator and the endpoint being validated. I am hopeful such a scheme may be useful for future deployments down the road. I think it is likely before there is infrastructure in place utilizing a new mechanism of that kind that current needs will need to be met with one of the other mechanisms. The speed and resource with which Let's Encrypt is working on solutions to migrate users to non-TLS-SNI validations might well be a signal.

will_hughes 9y ago

> Nothing says I control the domain like the ability to add and remove data from the authoritative DNS servers for the domain label (and children thereof) in question. Unfortunately this makes life difficult for us. We run a whitelabelled platform with tens of thousands of users - it's hard enough to get many people to understand setting a CNAME record (vs just setting an A record). Requiring them to update a TXT record once would be a big enough challenge, but doing it every 30-90 days is never going to happen. The smaller customers would probably be happy with us hosting their DNS, but we're not going to do that - the larger ones wouldn't. TLS-SNI being gone and DNS being unworkable means we're left with HTTP only.

mdhardeman 9y ago
But if you're a web post with 10k+ users, what's the problem with the HTTP-01 challenge? You just allow .well-known/* to be passed on to reflect the challenge responses you've generated for the client, while 301 redirecting everything else to their https:// https:// site. I'm confused how that would be harder for a web host at that scale? EDIT: I get people trying to run a server off their cable modem / rtr public IP, and 80 might be taken by something other than the target the port forward for 443 is going to -- and that's a problem for those use cases -- but that kind of concern wouldn't exist in a significant hosting infrastructure.
- will_hughes 9y ago
  > what's the problem with the HTTP-01 challenge? Nothing, yet. But who's to say that another similar bug won't be found in common shared-hosting platforms that forces LE to turn that challenge off too?
  - mdhardeman 9y ago
    True. Because almost all of the CAs utilize a web control mechanism, with many of them probably having processes not as rigorous as HTTP-01, it is likely that there would be significant backlash and a lengthier migration away from the method for that case. That said, anyone who can would be well advised to figure out how their DNS based mechanism would work if it were ever needed. As I and others have pointed out, there are clever and fully supported hacks for validating dns-01 without dynamic control of the full domain zone. (CNAME to another zone for the _acme-challenge labels, NS delegation to refer each _acme-challenge label as an independent zone at a different NS, etc.)