4 ms·

Wonder if services like Let’s Encrypt were affected. I imagine a scenario where a small hijack of DNS could allow for properly signed certificates for domains t

Wonder if services like Let’s Encrypt were affected. I imagine a scenario where a small hijack of DNS could allow for properly signed certificates for domains that are not owned. If I operated a CA service, I would carefully examine the requests received during this time frame. Maybe someone can audit the Transparency Logs during this period for anomalous activity.

erulabs 8y ago
I would assume they go far out of their way to resolve the address from a large number of distributed DNS resolvers - the question is if they're just round robin instead of multiple simultaneous queries, but I would hope that's part of why the DNS test usualy takes >60 seconds. If they're not treating DNS as a consensus protocol instead of as an immutable database, they'll always be open to poisoning attacks. If any certs were issued for hijacked domains (which as far as I've ready was only one, not using LetsEncrypt), it's a pretty glaring failure on the issuer, assuming they used "DNS Validation"
- btown 8y ago
  Per https://github.com/letsencrypt/boulder/blob/master/bdns/dns.go#L260 https://github.com/letsencrypt/boulder/blob/master/bdns/dns.... it seems they round robin. But they are aware of the issue in the spec: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.10.2 https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.htm... - “Querying the DNS from multiple vantage points to address local attackers” is a mentioned mitigation that a server could implement. Seems like a reasonable basis for a pull request.
- dexterdog 8y ago
  But on a big, distributed site the DNS is going to vary greatly depending upon the location of the query.
- mnordhoff 8y ago
  > why the DNS test usualy takes >60 seconds The server-side part of DNS validation takes about a second. The delay is all about clients waiting for their authoritative DNS servers to update. If you use a fast DNS provider, there's no reason to wait longer than necessary. > If any certs were issued for hijacked domains (which as far as I've ready was only one, not using LetsEncrypt), it's a pretty glaring failure on the issuer, assuming they used "DNS Validation" It wouldn't be a compliance failure, though. CAs are not required to be invulnerable to BGP hijacking attacks.
- peterwwillis 8y ago
  It doesn't matter how many DNS resolvers you have. If you can spoof BGP, and use it to spoof the DNS resolvers, and the authoritative name server, you can stand up your own DNS that says anything you want it to say. The only thing you cannot do is fake out DNSSEC. If you use a TLD which supports DNSSEC, you can make sure records have to be signed with your key. If fake DNS records aren't signed with your key, and the CA uses a validating stub resolver, and the CA is checking for CAA records, they will reject the attempt to create the cert. In theory. Here's the thing. The CAs know about these flaws and are still not providing more secure ways to prevent these attacks. How could it be more secure? The domain registrar could accept a public key from you at the time of purchase. At that point, nobody should ever be allowed to generate a cert anywhere without you signing off on it with your key. Is this some magically impossible technical challenge which mere mortals can't comprehend? Hell no. They literally just need to write an ASCII file in a database next to your domain name. But we aren't demanding it of them, so they aren't doing it. So attacks like this leave everyone vulnerable, because nobody cares enough to fix it.
  - cjbprime 8y ago
    And if you lose the key, as thousands of people would every day at a large registrar?
    - peterwwillis 8y ago
      Use the same method you use to pay for the domain to reset the key. This would require logging into your account at the registrar, or contacting support and providing enough verifiable material to reset it. If the registrar signed a message that had your key in it, they could re-sign it with a replacement key. So now you can 1) verify who originally assigned the domain, 2) record the public key, and 3) see a verifiable history of changes as long as the old history is included in successive signing. This could be used as part of a public record ala Certificate Transparency, or a blockchain-style system. If the attacker spoofs BGP and SE's your registrar, you (or whomever) can see if anyone creates a new key.
- jaas 8y ago
  Let's Encrypt does not currently validate from multiple perspectives but we're working on it. We're cooperating with a research team at Princeton to make sure our strategy is an effective mitigation against BGP attacks. Any given production validation right now may come from one of two of our datacenter locations, but not both. You may see multiple validation requests if you use our staging servers, but that's just an early version of multi perspective validation which doesn't work well enough to promote to production. There are some performance issues, and the external validation points are all on AWS, which uses the same autonomous system (AS) across datacenters. In order to effectively defend against BGP attacks we're probably going to have to deploy validation servers on 3+ different ASes so that our network perspectives are sufficiently diverse. If multi perspective validation is effective and nothing is done to significantly improve BGP security (we're not holding our breath) then I expect multi perspective validation to be a requirement for all CAs down the line.
  - peterwwillis 8y ago
    I'm trying really hard here not to be snide, but it is amazing to me that an organization that is responsible for securing the world wide web is basing that security on the hope that nobody can spoof 3 AS's at once. Just give up on BGP. Strongly suggest people use DNSSEC. For TLDs that don't support DNSSEC, require a public key issued to the registrar. You (the Certificate Authority) can get the public key from [R]WHOIS, IRIS, RDAP, whatever method you like, directly from the registrar. This is standard practice using the thin WHOIS data model, where .com and .net require domain registrars to maintain their own customers' data. Other TLDs simply run thick WHOIS servers that store all the data for their domains. Registrars are already required to pass up Delegation Signer records to a TLD to support DNSSEC. Passing on a similar record to a Certificate Authority would be practically the exact same thing. So we know the registrars can support it, and we know it would ensure that people would only be able to generate certificates for those domains that they actually own (and not "whomever can control the IP space currently").
    - pawal 8y ago
      We should not "give up on BGP". What we should do is to improve security in all layers. This includes BGP, and as you mention DNS. DNSSEC should be mandatory, just as TLS, for any business that take themselves seriously.
solatic 8y ago
Isn't this exactly the problem that EV certificates solve? By going the extra mile to validate identity instead of just trusting DNS ownership? Not that there's an easy way for LE to validate identity automatically - governments need to do a better job of providing electronic/smart identification, in which case a physical chip ID serves as a second factor (alongside DNS ownership) for automatic EV certs - but the weaknesses of only having DNS ownership are pretty much why EV certs exist.
technion 8y ago
I'm going to assume that Lets Encrypt wasn't impacted for the reason that there are several discussions that the hijacked myetherwallet website apparently had a self signed SSL cert running - and people were actually clicking through the warnings before handing over passwords. Surely they would have realised they could make the whole thing a lot more profitable if any SSL provider was impacted.
- kchr 8y ago
  But if you route all traffic for a specific domain through your own web server, surely you could complete the Letsencrypt verification steps as well. They just check for a specific file on the remote web server, right?
  - technion 8y ago
    Yes, but they weren't routing traffic for that web server to their own server. They were routing the IP of the DNS servers to their own server, and then just handing out the DNS address that suited them. In turn, if your own DNS wasn't configured to use a DNS server with a poisoned fraudulent address, a web server based verification landed on the valid server, not the attackers.
    - kchr 8y ago
      Okay, thanks for clarifying!

erulabs 8y ago

I would assume they go far out of their way to resolve the address from a large number of distributed DNS resolvers - the question is if they're just round robin instead of multiple simultaneous queries, but I would hope that's part of why the DNS test usualy takes >60 seconds. If they're not treating DNS as a consensus protocol instead of as an immutable database, they'll always be open to poisoning attacks. If any certs were issued for hijacked domains (which as far as I've ready was only one, not using LetsEncrypt), it's a pretty glaring failure on the issuer, assuming they used "DNS Validation"

btown 8y ago
Per https://github.com/letsencrypt/boulder/blob/master/bdns/dns.go#L260 https://github.com/letsencrypt/boulder/blob/master/bdns/dns.... it seems they round robin. But they are aware of the issue in the spec: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.10.2 https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.htm... - “Querying the DNS from multiple vantage points to address local attackers” is a mentioned mitigation that a server could implement. Seems like a reasonable basis for a pull request.
dexterdog 8y ago
But on a big, distributed site the DNS is going to vary greatly depending upon the location of the query.
mnordhoff 8y ago
> why the DNS test usualy takes >60 seconds The server-side part of DNS validation takes about a second. The delay is all about clients waiting for their authoritative DNS servers to update. If you use a fast DNS provider, there's no reason to wait longer than necessary. > If any certs were issued for hijacked domains (which as far as I've ready was only one, not using LetsEncrypt), it's a pretty glaring failure on the issuer, assuming they used "DNS Validation" It wouldn't be a compliance failure, though. CAs are not required to be invulnerable to BGP hijacking attacks.
peterwwillis 8y ago
It doesn't matter how many DNS resolvers you have. If you can spoof BGP, and use it to spoof the DNS resolvers, and the authoritative name server, you can stand up your own DNS that says anything you want it to say. The only thing you cannot do is fake out DNSSEC. If you use a TLD which supports DNSSEC, you can make sure records have to be signed with your key. If fake DNS records aren't signed with your key, and the CA uses a validating stub resolver, and the CA is checking for CAA records, they will reject the attempt to create the cert. In theory. Here's the thing. The CAs know about these flaws and are still not providing more secure ways to prevent these attacks. How could it be more secure? The domain registrar could accept a public key from you at the time of purchase. At that point, nobody should ever be allowed to generate a cert anywhere without you signing off on it with your key. Is this some magically impossible technical challenge which mere mortals can't comprehend? Hell no. They literally just need to write an ASCII file in a database next to your domain name. But we aren't demanding it of them, so they aren't doing it. So attacks like this leave everyone vulnerable, because nobody cares enough to fix it.
- cjbprime 8y ago
  And if you lose the key, as thousands of people would every day at a large registrar?
  - peterwwillis 8y ago
    Use the same method you use to pay for the domain to reset the key. This would require logging into your account at the registrar, or contacting support and providing enough verifiable material to reset it. If the registrar signed a message that had your key in it, they could re-sign it with a replacement key. So now you can 1) verify who originally assigned the domain, 2) record the public key, and 3) see a verifiable history of changes as long as the old history is included in successive signing. This could be used as part of a public record ala Certificate Transparency, or a blockchain-style system. If the attacker spoofs BGP and SE's your registrar, you (or whomever) can see if anyone creates a new key.
jaas 8y ago
Let's Encrypt does not currently validate from multiple perspectives but we're working on it. We're cooperating with a research team at Princeton to make sure our strategy is an effective mitigation against BGP attacks. Any given production validation right now may come from one of two of our datacenter locations, but not both. You may see multiple validation requests if you use our staging servers, but that's just an early version of multi perspective validation which doesn't work well enough to promote to production. There are some performance issues, and the external validation points are all on AWS, which uses the same autonomous system (AS) across datacenters. In order to effectively defend against BGP attacks we're probably going to have to deploy validation servers on 3+ different ASes so that our network perspectives are sufficiently diverse. If multi perspective validation is effective and nothing is done to significantly improve BGP security (we're not holding our breath) then I expect multi perspective validation to be a requirement for all CAs down the line.
- peterwwillis 8y ago
  I'm trying really hard here not to be snide, but it is amazing to me that an organization that is responsible for securing the world wide web is basing that security on the hope that nobody can spoof 3 AS's at once. Just give up on BGP. Strongly suggest people use DNSSEC. For TLDs that don't support DNSSEC, require a public key issued to the registrar. You (the Certificate Authority) can get the public key from [R]WHOIS, IRIS, RDAP, whatever method you like, directly from the registrar. This is standard practice using the thin WHOIS data model, where .com and .net require domain registrars to maintain their own customers' data. Other TLDs simply run thick WHOIS servers that store all the data for their domains. Registrars are already required to pass up Delegation Signer records to a TLD to support DNSSEC. Passing on a similar record to a Certificate Authority would be practically the exact same thing. So we know the registrars can support it, and we know it would ensure that people would only be able to generate certificates for those domains that they actually own (and not "whomever can control the IP space currently").
  - pawal 8y ago
    We should not "give up on BGP". What we should do is to improve security in all layers. This includes BGP, and as you mention DNS. DNSSEC should be mandatory, just as TLS, for any business that take themselves seriously.

solatic 8y ago

Isn't this exactly the problem that EV certificates solve? By going the extra mile to validate identity instead of just trusting DNS ownership? Not that there's an easy way for LE to validate identity automatically - governments need to do a better job of providing electronic/smart identification, in which case a physical chip ID serves as a second factor (alongside DNS ownership) for automatic EV certs - but the weaknesses of only having DNS ownership are pretty much why EV certs exist.

technion 8y ago

I'm going to assume that Lets Encrypt wasn't impacted for the reason that there are several discussions that the hijacked myetherwallet website apparently had a self signed SSL cert running - and people were actually clicking through the warnings before handing over passwords. Surely they would have realised they could make the whole thing a lot more profitable if any SSL provider was impacted.

kchr 8y ago
But if you route all traffic for a specific domain through your own web server, surely you could complete the Letsencrypt verification steps as well. They just check for a specific file on the remote web server, right?
- technion 8y ago
  Yes, but they weren't routing traffic for that web server to their own server. They were routing the IP of the DNS servers to their own server, and then just handing out the DNS address that suited them. In turn, if your own DNS wasn't configured to use a DNS server with a poisoned fraudulent address, a web server based verification landed on the valid server, not the attackers.
  - kchr 8y ago
    Okay, thanks for clarifying!