9 ms·

Deep Inside a DNS Amplification DDoS Attack

tptacek 14y ago
While it's not the most compelling of the arguments against DNSSEC (which is an ineffective and unnecessary complication of the Internet core protocols), amplification has been the case Daniel J. Bernstein has been making against it for years. Here's a good intro: http://dnscurve.org/amplification.html http://dnscurve.org/amplification.html
- mike-cardwell 14y ago
  It's not a very compelling argument against DNSSEC no. The solution is for the World to close down open DNS servers in the same way it did with open SMTP servers. Granted, it was a lot easier to do with SMTP.
  - tptacek 14y ago
    Or, we could simply not do DNSSEC. DNSSEC + many tens of millions of dollars of open redirector cleanup work is not an inherently better solution than simply not doing DNSSEC.
    - mike-cardwell 14y ago
      DNSSEC does not cause DNS amplification attacks. It just makes them worse. If we want to stop DNS amplification attacks, "not doing DNSSEC" isn't the fix. Closing down open redirectors is. If we turned off DNSSEC tomorrow, we'd still see these attacks.
      - tptacek 14y ago
        DNSSEC doesn't cause DNS amplification attacks. It just makes them much, much worse. Meanwhile, DNSSEC itself provides minimal value (all online commerce on the Internet happens without DNSSEC today, and to a useful first approximation, none of today's online fraud depends on spoofing DNS).
        mike-cardwell 14y ago
        Except that DNS needs to be secure before systems that rely on DNS being secure, are written.
        tptacek 14y ago
        Three issues with that, from least to most important --- that is, (c) is the most important issue: (a) DNSSEC doesn't actually do a good job of making the DNS secure (for instance, it doesn't secure the "last mile" between desktops and recursive resolvers). (b) Securing the DNS doesn't automatically secure the other core protocols that also need to be secure to rely on DNS promises; an IP address is still just an insecure IP address even if you learn about it from an RSA-signed message. (c) There's no compelling case that the DNS needs to be secure, or that we need systems that rely on a secure DNS. A secure DNS doesn't make it any easier to clear credit card transactions, to transfer funds, or to create secure & anonymous messaging systems. It seems to me that if DNS security was such a serious problem, it would be easier to come up with a scenario that benefited from it; that is, you wouldn't need to handwave with a comment like "the DNS needs to be secure before systems that rely on DNS being secure are written". Well, sure, but that shouldn't make it hard to merely imagine such a system. What is it?
        mike-cardwell 14y ago
        You've convinced me. There is no conceivable benefit from a secure DNS, nor any possible application that could make use of such a thing.
        tptacek 14y ago
        I wish I felt like you were being sincere, since that actually is one of my arguments!
        mike-cardwell 14y ago
        I was being sarcastic because I (and everybody reading this comment, other than you apparently) could probably real off at least fifty use cases given an hour, so I found it hilarious that you were claiming that I couldn't even name one. I am not going to name one. You may now repeat your claim that the reason is because I can't think of one, for continued comedy affect.
    - peterwwillis 14y ago
      This seems like a stupid solution. Did they fix the FTP bounce attack by disabling all PORT commands forever? No. They disabled PORT being used for any host except the originator. Similarly, they could compromise here by only enabling DNSSEC for authorized clients, and keep the rest of the DNS records open for the world the way they work now. This of course does nothing to fix the pink elephant of protocols that rely on a source address in a UDP packet to shovel off data without any limits. DNSSEC is just one single feature that can be abused; i'm sure there are many more available in other protocols, and more yet to be invented.
      - tptacek 14y ago
        It would be a stupid solution if DNSSEC had some vital role to play in Internet security. But it clearly doesn't, as evidenced by the fact that virtually all high-volume commercial transactions conducted in the industrialized world in 2012 hit the Internet, and none of it is protected by DNSSEC. If I had to draw a pie chart explaining the rationale behind deploying DNSSEC, a 1/3 slice of that pie would be labeled "IETF's misguided effort to replace the broken TLS CA PKI with yet another PKI controlled largely by the same giant businesses", and the remaining 2/3 slice would be labeled "Self-perpetuating fallacy that DNS must be SEC'd the way IP was (unsuccessfully) SEC'd", or, less charitably, "Windmill tilting exercise on the part of standards bodies".
        peterwwillis 14y ago
        I don't care about DNSSEC. I just think it's ridiculous to ignore the real flaw, which is that you can bounce DNS packets at anyone you want. If you want to strip away part of the protocol to fix an attack vector, why not just redact EDNS0? Or do what networks around the world already do and block all port 53 udp packets bigger than 512 bytes. We'll continue to have a size-limited and somewhat inflexible protocol, but at least DNS amplification will have an upper bound.
        tptacek 14y ago
        You can't just wave a magic wand and make it so attackers can't bounce DNS packets, and so it actually does matter than one DNS feature makes those bounced packets much much larger than any other DNS feature.
- morsch 14y ago
  Kaminsky on DNSSEC amplification: http://dankaminsky.com/2011/01/05/djb-ccc/#dnsamp http://dankaminsky.com/2011/01/05/djb-ccc/#dnsamp He doesn't think it's a very strong argument against DNSSEC.
  - tptacek 14y ago
    This post doesn't actually say anything. It says that DNSSEC amplification is a known attack --- nobody is suggesting Bernstein "invented it" --- and that there are other amplification vectors besides DNSSEC. The latter point would be compelling if those vectors were as powerful as DNSSEC, but they aren't. The notion that the problem is "actually open redirectors" (nameservers configured to answer queries from arbitrary points on the Internet) is indicative of the weird reasoning that the IETF DNS people have used all throughout the DNSSEC process. Open redirectors means DNSSEC is a viable mechanism to get ISPs to flood random sites off the Internet? Just mandate that ISPs not run DNSSEC that way! Secret DNS names mean that verified negative answers in the DNSSEC protocol will breach confidentiality? Just mandate that nobody have secret DNS names! Daniel J. Bernstein did a much more convincing takedown of dakami's reasoning in a talk at 27C3; I'm not going to recap it. I'd just say Daniel J. Bernstein has earned the authority he speaks with regarding DNS security; the vulnerability dakami is famous for discovering is one that djbdns --- released many years before that vulnerability was disclosed --- was designed in part to address. Obviously (if you've ever installed djbdns), Bernstein did a good job of handling the open redirector problem as well. Kaminsky got on the wrong side of this issue, which is ironic, because he's put a lot more time into practical DNS security than the people he's arguing on behalf of.
    - morsch 14y ago
      Hmm. What I got was that, disregarding the open redirector thing, DNSSEC is typically "only" about two times worse than regular DNS amplification. DJB's number is a lot higher.
      - tptacek 14y ago
        First, Bernstein is talking about servers, and Kaminsky is shifting the goalposts to caches. Second, you can compare their numbers directly (get Bernstein's from any of his DNS talks at cr.yp.to). Third, read closely and you run into things in Kaminsky's post like this: That’s a 3.6KB response to a 64 byte request, no DNSSEC required. I’ve been saying this for a while: DNSSEC is just DNS with signatures. Whose bug is it anyway? Well, at least some of those servers are running DJB’s dnscache… Well, probably not, because dnscache was, from the time of its release, the first cache server to ship default-deny for remote queries; if you want dnscache to serve as an open cache, you have to jump through hoops to configure it that way. Most open cache servers are BIND.
- dchest 14y ago
  Video of djb's talk which mentions DNSSEC amplification: http://vimeo.com/18417770 http://vimeo.com/18417770 Slides for this talk: http://cr.yp.to/talks/2010.12.28/slides.pdf http://cr.yp.to/talks/2010.12.28/slides.pdf (Also http://cr.yp.to/talks.html http://cr.yp.to/talks.html Ctrl+F "DNSSEC")
  - tptacek 14y ago
    It's a good talk! He's a good speaker.
latchkey 14y ago
Why doesn't BIND disable/block open recursion by default? It should be a configuration option that you have to turn on and should spit out warning messages in the logs.
- WorldGroove 14y ago
  I run a game server that requires any IP to be able to use the DNS server I have set up. Similar to the Pokemon hacks out there. I don't want to turn recursion off, because I want people to continue using their game-console to go to any website. However, I have iptables rules set up that pretty much -j DROP anything that isn't query type "IN A" or "IN AAAA" So far, all the DDoS I've gotten in my logs involved an "IN ANY" request.
joe_bleau 14y ago
My little personal VPS (non-recursive!) DNS server has been asked to participate in one of these DDoS attacks before; the incoming requests for isc.org tipped me off. I guess they didn't notice that my server wasn't sending replies.
otterley 14y ago
In addition, it's network engineers' responsibility to prevent source address spoofing by dropping all outgoing packets having a source address that's not on a connected subnet.
- danielweber 14y ago
  What kind of connection do you need to the Internet to spoof the source address these days? I.e., who isn't egress filtering?
  - entropy_ 14y ago
    I was wondering the same thing. I live in a third world country and I've never seen a connection with no egress filtering.
- aidenn0 14y ago
  I have wondered about this; does anyone who knows more about networks than me know why source-address spoofing is still alive and well when it was a known issue at least as far back as the late '90s (when I first learned about smurf attacks). In particular since most DDoS attacks originate from botnets, simply egress filtering at the ISP level should be sufficient.
  - grandinj 14y ago
    Laziness on the part of the network operators. Seriously, they're just too lazy to auto-generate firewall rules from their list of assigned addresses.
    - pfortuny 14y ago
      Nope: someone will come up and say "network neutrality" and believe he would be right in this case.
    - deleted 14y ago
      [deleted]
    - 14y ago
agwa 14y ago
I was curious how operators of public DNS resolvers, such as Google, prevent themselves from becoming unwitting aids to DNS Amplification attacks. I found this info about Google's resolver: https://developers.google.com/speed/public-dns/docs/security https://developers.google.com/speed/public-dns/docs/security The tl;dr is rate limiting, plus some other techniques like adaptively restricting the ratio of request size to response size.
donavanm 14y ago
A bit disappointed that the truncate bit wasn't mentioned. So, for the open resolver owners playing along at home: Instead of dropping suspect requests send the truncate bit back. There's no byte amplification in the response, which defeats the attacks raison d'etre. By dropping the incoming packets you're punishing well behaved resolvers. They're going to take a 2-500ms latency hit before retrying another resolver or over tcp. A well behaved resolver will respond to a truncate by sending the same query over tcp. That's only a latency hit of the rtt. The tcp connection also effectively authenticates control of that source ip address. Now add that src ip to your whitelist of known good resolvers. Attacker mitigated, other customers not impacted.
- gridspy 14y ago
  Bear in mind that none of these DNS queries are ever sent out of CloudFlare's network - as mentioned in the previous blog entry, none of these incoming responses are valid. See http://blog.cloudflare.com/65gbps-ddos-no-problem http://blog.cloudflare.com/65gbps-ddos-no-problem " What's great is that we can safely respond and ask them to block all DNS requests originating from our network since our IPs should never originate a DNS request to a resolver. "
  - donavanm 14y ago
    Please to be rereading my comment. It's addressed to anyone running a resolver/authoritative name server, not the attack target. I promise I'm well acquainted with spoofed DNS attacks.
follower 14y ago
I encountered a DNS Amplification attack recently and was confused how the VM I was using could be involved given it wasn't running BIND. It turned out that a VPN was implemented using `dnsmasq` which was responding to the DNS queries. I ended up using firewall rules to drop the queries because it seems like in certain situations it's not enough to just configure `dnsmasq` to not respond to the requests: http://people.canonical.com/~ubuntu-security/cve/2012/CVE-2012-3411.html http://people.canonical.com/~ubuntu-security/cve/2012/CVE-20... Just incase the information is useful to anyone else. :) (I Am Not A Network Engineer.)

While it's not the most compelling of the arguments against DNSSEC (which is an ineffective and unnecessary complication of the Internet core protocols), amplification has been the case Daniel J. Bernstein has been making against it for years. Here's a good intro: http://dnscurve.org/amplification.html http://dnscurve.org/amplification.html

mike-cardwell 14y ago
It's not a very compelling argument against DNSSEC no. The solution is for the World to close down open DNS servers in the same way it did with open SMTP servers. Granted, it was a lot easier to do with SMTP.
- tptacek 14y ago
  Or, we could simply not do DNSSEC. DNSSEC + many tens of millions of dollars of open redirector cleanup work is not an inherently better solution than simply not doing DNSSEC.
  - mike-cardwell 14y ago
    DNSSEC does not cause DNS amplification attacks. It just makes them worse. If we want to stop DNS amplification attacks, "not doing DNSSEC" isn't the fix. Closing down open redirectors is. If we turned off DNSSEC tomorrow, we'd still see these attacks.
    - tptacek 14y ago
      DNSSEC doesn't cause DNS amplification attacks. It just makes them much, much worse. Meanwhile, DNSSEC itself provides minimal value (all online commerce on the Internet happens without DNSSEC today, and to a useful first approximation, none of today's online fraud depends on spoofing DNS).
      - mike-cardwell 14y ago
        Except that DNS needs to be secure before systems that rely on DNS being secure, are written.
        tptacek 14y ago
        Three issues with that, from least to most important --- that is, (c) is the most important issue: (a) DNSSEC doesn't actually do a good job of making the DNS secure (for instance, it doesn't secure the "last mile" between desktops and recursive resolvers). (b) Securing the DNS doesn't automatically secure the other core protocols that also need to be secure to rely on DNS promises; an IP address is still just an insecure IP address even if you learn about it from an RSA-signed message. (c) There's no compelling case that the DNS needs to be secure, or that we need systems that rely on a secure DNS. A secure DNS doesn't make it any easier to clear credit card transactions, to transfer funds, or to create secure & anonymous messaging systems. It seems to me that if DNS security was such a serious problem, it would be easier to come up with a scenario that benefited from it; that is, you wouldn't need to handwave with a comment like "the DNS needs to be secure before systems that rely on DNS being secure are written". Well, sure, but that shouldn't make it hard to merely imagine such a system. What is it?
        mike-cardwell 14y ago
        You've convinced me. There is no conceivable benefit from a secure DNS, nor any possible application that could make use of such a thing.
        tptacek 14y ago
        I wish I felt like you were being sincere, since that actually is one of my arguments!
        mike-cardwell 14y ago
        I was being sarcastic because I (and everybody reading this comment, other than you apparently) could probably real off at least fifty use cases given an hour, so I found it hilarious that you were claiming that I couldn't even name one. I am not going to name one. You may now repeat your claim that the reason is because I can't think of one, for continued comedy affect.
  - peterwwillis 14y ago
    This seems like a stupid solution. Did they fix the FTP bounce attack by disabling all PORT commands forever? No. They disabled PORT being used for any host except the originator. Similarly, they could compromise here by only enabling DNSSEC for authorized clients, and keep the rest of the DNS records open for the world the way they work now. This of course does nothing to fix the pink elephant of protocols that rely on a source address in a UDP packet to shovel off data without any limits. DNSSEC is just one single feature that can be abused; i'm sure there are many more available in other protocols, and more yet to be invented.
    - tptacek 14y ago
      It would be a stupid solution if DNSSEC had some vital role to play in Internet security. But it clearly doesn't, as evidenced by the fact that virtually all high-volume commercial transactions conducted in the industrialized world in 2012 hit the Internet, and none of it is protected by DNSSEC. If I had to draw a pie chart explaining the rationale behind deploying DNSSEC, a 1/3 slice of that pie would be labeled "IETF's misguided effort to replace the broken TLS CA PKI with yet another PKI controlled largely by the same giant businesses", and the remaining 2/3 slice would be labeled "Self-perpetuating fallacy that DNS must be SEC'd the way IP was (unsuccessfully) SEC'd", or, less charitably, "Windmill tilting exercise on the part of standards bodies".
      - peterwwillis 14y ago
        I don't care about DNSSEC. I just think it's ridiculous to ignore the real flaw, which is that you can bounce DNS packets at anyone you want. If you want to strip away part of the protocol to fix an attack vector, why not just redact EDNS0? Or do what networks around the world already do and block all port 53 udp packets bigger than 512 bytes. We'll continue to have a size-limited and somewhat inflexible protocol, but at least DNS amplification will have an upper bound.
        tptacek 14y ago
        You can't just wave a magic wand and make it so attackers can't bounce DNS packets, and so it actually does matter than one DNS feature makes those bounced packets much much larger than any other DNS feature.
morsch 14y ago
Kaminsky on DNSSEC amplification: http://dankaminsky.com/2011/01/05/djb-ccc/#dnsamp http://dankaminsky.com/2011/01/05/djb-ccc/#dnsamp He doesn't think it's a very strong argument against DNSSEC.
- tptacek 14y ago
  This post doesn't actually say anything. It says that DNSSEC amplification is a known attack --- nobody is suggesting Bernstein "invented it" --- and that there are other amplification vectors besides DNSSEC. The latter point would be compelling if those vectors were as powerful as DNSSEC, but they aren't. The notion that the problem is "actually open redirectors" (nameservers configured to answer queries from arbitrary points on the Internet) is indicative of the weird reasoning that the IETF DNS people have used all throughout the DNSSEC process. Open redirectors means DNSSEC is a viable mechanism to get ISPs to flood random sites off the Internet? Just mandate that ISPs not run DNSSEC that way! Secret DNS names mean that verified negative answers in the DNSSEC protocol will breach confidentiality? Just mandate that nobody have secret DNS names! Daniel J. Bernstein did a much more convincing takedown of dakami's reasoning in a talk at 27C3; I'm not going to recap it. I'd just say Daniel J. Bernstein has earned the authority he speaks with regarding DNS security; the vulnerability dakami is famous for discovering is one that djbdns --- released many years before that vulnerability was disclosed --- was designed in part to address. Obviously (if you've ever installed djbdns), Bernstein did a good job of handling the open redirector problem as well. Kaminsky got on the wrong side of this issue, which is ironic, because he's put a lot more time into practical DNS security than the people he's arguing on behalf of.
  - morsch 14y ago
    Hmm. What I got was that, disregarding the open redirector thing, DNSSEC is typically "only" about two times worse than regular DNS amplification. DJB's number is a lot higher.
    - tptacek 14y ago
      First, Bernstein is talking about servers, and Kaminsky is shifting the goalposts to caches. Second, you can compare their numbers directly (get Bernstein's from any of his DNS talks at cr.yp.to). Third, read closely and you run into things in Kaminsky's post like this: That’s a 3.6KB response to a 64 byte request, no DNSSEC required. I’ve been saying this for a while: DNSSEC is just DNS with signatures. Whose bug is it anyway? Well, at least some of those servers are running DJB’s dnscache… Well, probably not, because dnscache was, from the time of its release, the first cache server to ship default-deny for remote queries; if you want dnscache to serve as an open cache, you have to jump through hoops to configure it that way. Most open cache servers are BIND.
dchest 14y ago
Video of djb's talk which mentions DNSSEC amplification: http://vimeo.com/18417770 http://vimeo.com/18417770 Slides for this talk: http://cr.yp.to/talks/2010.12.28/slides.pdf http://cr.yp.to/talks/2010.12.28/slides.pdf (Also http://cr.yp.to/talks.html http://cr.yp.to/talks.html Ctrl+F "DNSSEC")
- tptacek 14y ago
  It's a good talk! He's a good speaker.

latchkey 14y ago

Why doesn't BIND disable/block open recursion by default? It should be a configuration option that you have to turn on and should spit out warning messages in the logs.

WorldGroove 14y ago
I run a game server that requires any IP to be able to use the DNS server I have set up. Similar to the Pokemon hacks out there. I don't want to turn recursion off, because I want people to continue using their game-console to go to any website. However, I have iptables rules set up that pretty much -j DROP anything that isn't query type "IN A" or "IN AAAA" So far, all the DDoS I've gotten in my logs involved an "IN ANY" request.

joe_bleau 14y ago

My little personal VPS (non-recursive!) DNS server has been asked to participate in one of these DDoS attacks before; the incoming requests for isc.org tipped me off. I guess they didn't notice that my server wasn't sending replies.

otterley 14y ago

In addition, it's network engineers' responsibility to prevent source address spoofing by dropping all outgoing packets having a source address that's not on a connected subnet.

danielweber 14y ago
What kind of connection do you need to the Internet to spoof the source address these days? I.e., who isn't egress filtering?
- entropy_ 14y ago
  I was wondering the same thing. I live in a third world country and I've never seen a connection with no egress filtering.
aidenn0 14y ago
I have wondered about this; does anyone who knows more about networks than me know why source-address spoofing is still alive and well when it was a known issue at least as far back as the late '90s (when I first learned about smurf attacks). In particular since most DDoS attacks originate from botnets, simply egress filtering at the ISP level should be sufficient.
- grandinj 14y ago
  Laziness on the part of the network operators. Seriously, they're just too lazy to auto-generate firewall rules from their list of assigned addresses.
  - pfortuny 14y ago
    Nope: someone will come up and say "network neutrality" and believe he would be right in this case.
  - deleted 14y ago
    [deleted]
  - 14y ago

agwa 14y ago

I was curious how operators of public DNS resolvers, such as Google, prevent themselves from becoming unwitting aids to DNS Amplification attacks. I found this info about Google's resolver: https://developers.google.com/speed/public-dns/docs/security https://developers.google.com/speed/public-dns/docs/security The tl;dr is rate limiting, plus some other techniques like adaptively restricting the ratio of request size to response size.

donavanm 14y ago

A bit disappointed that the truncate bit wasn't mentioned. So, for the open resolver owners playing along at home: Instead of dropping suspect requests send the truncate bit back. There's no byte amplification in the response, which defeats the attacks raison d'etre. By dropping the incoming packets you're punishing well behaved resolvers. They're going to take a 2-500ms latency hit before retrying another resolver or over tcp. A well behaved resolver will respond to a truncate by sending the same query over tcp. That's only a latency hit of the rtt. The tcp connection also effectively authenticates control of that source ip address. Now add that src ip to your whitelist of known good resolvers. Attacker mitigated, other customers not impacted.

gridspy 14y ago
Bear in mind that none of these DNS queries are ever sent out of CloudFlare's network - as mentioned in the previous blog entry, none of these incoming responses are valid. See http://blog.cloudflare.com/65gbps-ddos-no-problem http://blog.cloudflare.com/65gbps-ddos-no-problem " What's great is that we can safely respond and ask them to block all DNS requests originating from our network since our IPs should never originate a DNS request to a resolver. "
- donavanm 14y ago
  Please to be rereading my comment. It's addressed to anyone running a resolver/authoritative name server, not the attack target. I promise I'm well acquainted with spoofed DNS attacks.

follower 14y ago

I encountered a DNS Amplification attack recently and was confused how the VM I was using could be involved given it wasn't running BIND. It turned out that a VPN was implemented using `dnsmasq` which was responding to the DNS queries. I ended up using firewall rules to drop the queries because it seems like in certain situations it's not enough to just configure `dnsmasq` to not respond to the requests: http://people.canonical.com/~ubuntu-security/cve/2012/CVE-2012-3411.html http://people.canonical.com/~ubuntu-security/cve/2012/CVE-20... Just incase the information is useful to anyone else. :) (I Am Not A Network Engineer.)