6 ms·

"The key feature is a built-in custom TCP/IP stack capable of handling millions of DNS queries-per-second per CPU core." DNS uses UDP primarily. I suspect the

• justinsb 12y ago
  Absolutely, but I would say that this has the potential to be a lot more secure than a traditional (full) TCP/IP stack. Most queries are UDP (one packet), and we would expect that TCP connections should only last a few packets. TCP connections that don't match a handful of patterns would be suspicious (IMHO) and should probably just be dropped. Of course, then you find some weird version of Windows XP that this breaks :-(
  • jvehent 12y ago
    > TCP connections that don't match a handful of patterns would be suspicious (IMHO) and should probably just be dropped. I used to believe that. It is unfortunately not true. TCP packets will come in all shapes and forms, and all must be treated equally, which is what makes TCP stacks so incredibly complex to implement. The Linux TCP stack is quite safe and fast. Especially with tight integration with NICs hardware (checksum offloading and the like). I'm a bit unsure what a custom stack in userland can provide that the standard kernel stacks don't have.
    • thirsteh 12y ago
      Performance. This isn't for security, or it wouldn't be yet another critical piece of software written in C.
    • tptacek 12y ago
      Can you be more specific about the different-but-equal forms TCP packets take? You can dive into the details; I've implemented TCP stacks too.
    • robertgraham 12y ago
      The Linux TCP stack is NOT fast. My DNS server can respond to DNS queries faster than the simplest of in-kernel echo servers (like ICMP ping or UDP port 7 echo). That's with the entire DNS overhead of parsing the DNS protocol, looking up the name in a very large database (like the .com zone), doing compression, and generating a response. The upshot is this: going through the Linux stack, a DNS server is limited to around 2 million queries/second. Using a custom user-mode stack, it can achieve 10 million queries per second.
      • modusponens 12y ago
        Do you have any insight about how other stacks fare on that same workload? (*bsd, qnx, ms-windows, minix,osx...)
  • nitinics 12y ago
    With the advent of DNSSEC, IPv6 and EDNS0 you're more likely to see DNS responses >512 bytes, therefore falling back to TCP (with truncate bit set). Therefore it is strongly recommended you do not drop/block tcp 53 on your middleboxes , firewalls etc.
    • justinsb 12y ago
      Sure; I'm not suggesting dropping TCP entirely, just that e.g. a 1MB request / response is not going to be legitimate, and so you can simply not implement a lot of TCP's complexity (e.g. window scaling)
    • colmmacc 12y ago
      EDNS0 includes a mechanism for clients/resolvers to signal that they can handle a large/fragmented UDP response. At this point about 85% of requesters can handle UDP responses of at least 4K. For the moment, DNSSEC and EDNS0 are making falling back to TCP far less common than it used to be. That may change, as some providers are starting to put smaller limits on their response sizes (to mitigate certain kinds of DDOS and response spoofing attacks). Of course permitting TCP 53 is required for DNS to work; as is permitting UDP fragments (which poorly configured firewalls often block too).
    • zaroth 12y ago
      Another interesting use-case for TCP in DNS is for anti-DDoS. If a botnet is abusing your DNS server to flood traffic to their target, flipping the 'TC' bit which will force the request to come back over TCP, exposing the spoof. My long-winded write-up here: http://opine.me/cert-advisory-on-dns-amplification-offers-little-hope/ http://opine.me/cert-advisory-on-dns-amplification-offers-li...
      • jamiesonbecker 12y ago
        Very interesting, thx for the link!
• dogma1138 12y ago
  TCP/IP is the common name for the Internet Protocol Suite which UDP is part of. Also since many mobile clients tend to issue requests over TCP (due to it being more reliable over mobile), and that many responses can be larger than 512 bytes now and EDNS is not really a standard DNS over TCP will probably over take UDP quite quickly. The text below is from the RFC, and no it does not relates to zone transfers but to normal DNS queries :) All general-purpose DNS implementations MUST support both UDP and TCP transport. o Authoritative server implementations MUST support TCP so that they do not limit the size of responses to what fits in a single UDP packet. o Recursive server (or forwarder) implementations MUST support TCP so that they do not prevent large responses from a TCP-capable server from reaching its TCP-capable clients. o Stub resolver implementations (e.g., an operating system's DNS resolution library) MUST support TCP since to do otherwise would limit their interoperability with their own clients and with upstream servers. Stub resolver implementations MAY omit support for TCP when specifically designed for deployment in restricted environments where truncation can never occur or where truncated DNS responses are acceptable. And for the most important part :P Regarding the choice of when to use UDP or TCP, Section 6.1.3.2 of RFC 1123 also says: ... a DNS resolver or server that is sending a non-zone-transfer query MUST send a UDP query first. That requirement is hereby relaxed. A resolver SHOULD send a UDP query first, but MAY elect to send a TCP query instead if it has good reason to expect the response would be truncated if it were sent over UDP (with or without EDNS0) or for other operational reasons, in particular, if it already has an open TCP connection to the server.
  • wging 12y ago
    This is RFC 5966, for those who wondered: http://tools.ietf.org/html/rfc5966 http://tools.ietf.org/html/rfc5966
  • tptacek 12y ago
    Why would TCP DNS overtake UDP "quite quickly" when it hasn't done so in the past decade? What's meaningfully changed about DNS recently?
    • 12y ago
• marktangotango 12y ago
  Seems like this user land tcp stack could be the basis of types of c10m servers. Are tbere existenting userland tcp stacks available? I'm not familiar with any.
  • wmf 12y ago
    http://shader.kaist.edu/mtcp/ http://shader.kaist.edu/mtcp/ http://www.openonload.org/ http://www.openonload.org/ There's also some work showing that you can achieve very high performance with kernel TCP: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/belay https://www.usenix.org/conference/osdi14/technical-sessions/...
  • tptacek 12y ago
    lwip would be a well-known example, right?
  • daurnimator 12y ago
    Check out snabb switch: http://www.snabb.co/ http://www.snabb.co/ https://github.com/SnabbCo/snabbswitch/wiki https://github.com/SnabbCo/snabbswitch/wiki
  • zzzcpan 12y ago
    Also take a look at NetBSD's rump kernels and PicoTCP. Nobody mentioned them.
  • robertgraham 12y ago
    I build stacks that are highly customized for the target solution and are impractical for general purpose use. A good general purpose stack is the 6windgate stack. I know nothing about it personally, but I know that a lot of people do use it successfully.
• cperciva 12y ago
  I wrote a basic TCP stack for a honeypot research project. It is hard and incredibly complex. Yes and no. Most of the complication comes from extra functionality (segmentation offload, checksum offload, SACK) or from functionality which is required by the standard but not relevant for a DNS resolver (congestion control, window management, TCP timers). If all you're doing is accepting a TCP connection, reading a small request, and writing a small response back, you can remove about 90% of the code from your TCP stack.
• tptacek 12y ago
  Could you be a little more specific about what's incredibly complex about writing an interoperable TCP stack? That aside: if I had to guess, this would be Robert Graham's 10th IP stack. He's been doing this (specifically) since the late 1990s.
  • colmmacc 12y ago
    I'm not the OP, but I think it's fair to call it complex, and I'd pick three requirements out in particular. 1. Path reachability, MTU discovery and MSS interaction When sending outbound packets, you have to correlate incoming ICMP error messages in case they signal a problem. If the problem is that the packet is too big, you have to figure out what the MTU really is (which can take repeated attempts), so that you know what MSS to use (for TCP, or fragmentation boundary for UDP). If the path is unreachable, you have to remember that too. In both cases, you need some kind of global book-keeping so that you can do the right thing across connections. Some protocols (like active FTP) implicitly rely on MTU discovery on one connection signaling the MSS for another connection, so everything has to be path based, rather than connection based. Messy. 2. State management for error correlation O.k., so you've figured out how to fragment an outgoing datagram and know what boundary to use, but how do you handle incoming error messages related to the fragments? Even for UDP, or other "stateless" protocols you actually do have to keep state so that you can correlate those error messages to the packets you sent. When the error message comes back, it will have the IP ID of the fragment, but nothing else is guaranteed. This goes for (1.) too, but ICMP error messages can also be recursive and nested, and for a correct implementation you need to consider how to handle ICMP error messages that were themselves triggered by ICMP error messages. Several userspace stacks get this wrong, and can't correctly handle MTU discovery for UDP, or double-error correlation. 3. Heuristical and inconsistent caps on state Many TCP implementations support selective acknowledgements and duplicate ack signalling, but what are their tolerances, just how much data can be retransmitted or handled out of order before you have the resend the whole window? there's no way to know, and if you get it wrong you can end up stalling a TCP connection for a significant delay. Unfortunately there are no simple limits, and in some cases the volumes are related to bandwidth delay products, necessitating some kind of integral control loop. The problem with all of these is that they only show up "sometimes" and with particular networks or TCP stacks. I've limited these to interoperability issues - but there are other tricky complexities. For example, when building a TCP stack, do you optimise for throughput and so batch reads/writes of many packets - or do you optimize for a correct RTT estimate, and do things more synchronously. It's not possible to have both (at least with today's NIC interfaces); sometimes RTT is critical (e.g. an NTP implementation, a real-time control system or just any system that needs to rapidly recover from packet loss) , sometimes throughput is more important. Definitely complex.