9 ms·

A toy DNS resolver

• NoWayDude1 5y ago
  [flagged]
  • taeric 5y ago
    More than a little unfair when the following is in the article: I’m not going to write this completely from scratch – I think parsing DNS packets is really interesting, but it’s definitely more than 80 lines of code, and I find that it kind of distracts from the algorithm. Which, seems legit to me. This article was a fun read to refresh on exactly what goes on in resolving basic records.
    • dixie_land 5y ago
      I agree it’s a fun article but you can’t deny the title is at the very least clickbait-ish
      • tptacek 5y ago
        I hereby do fully deny that there is anything the least bit clickbait-ish about this article, which is super interesting and all the more an achievement for its brevity. It demystifies what is the most annoying thing about trying to code the client side of DNS, and the fact that it uses the same DNS message parsing library every other Go programmer does isn't of the slightest import --- if the article hadn't used miekg/dns, some other lowbrow comment would be taking the author to task for reinventing the message codec wheel. The world needs more simple recursive lookup examples, and absolutely does not need more explication of DNS message parsing, any more than than an article talking about etcd's Raft implementation would benefit from a hand-rolled implementation of Protobufs. Yikes, everybody.
        • tedunangst 5y ago
          It's (sadly) telling that all the dismissals are focused on packet parsing, and not like, bailiwick checking, which is understandably missing, but also really important.
    • zamadatix 5y ago
      This is the key case I wish HN allowed editorializing the title. Regardless how good the article is (or isn't) when they have titles like this it's inevitable the conversation will be about how the title was overreaching. If people were encouraged to make titles like this into something like "Making a simple DNS resolver with miekg/dns in Go" or even just "A DNS resolver in Go" when posting we'd avoid this multiple times a day recurring problem. Or maybe it already is encouraged and people just aren't aware this counts as "misleading" or "linkbait" since it's not well described what the cutoff point is. Either way both titles like this one and discussions of have long gotten old. Edit: looks like the article moved to "A toy DNS resolver", excellent choice of wording IMO.
      • justsomehnguy 5y ago
        > This is the key case I wish HN allowed editorializing the title The ability to leave a short (less than 100? 140?) description would be one the ways which would allows not to clickba^W editorialize the titles. Or the submission author can do the same by leaving the comment about it in the first comment.
        • throwaway1777 5y ago
          Leaving description opens up other bad avenues though. Like posting an article with a description about how stupid the article or the author is.
          • zamadatix 5y ago
            The more general take I have seen is that it would give a "priority comment". That can result in problems like that but also more commonly just cause the comments to all act like a response to the "main" comment rather than general discourse on the topic. I can definitely see that though I also see the poster having a large amount of that control already in most cases since they get to pick the source they post.
          • airstrike 5y ago
            > Like posting an article with a description about how stupid the article or the author is. The community can moderate that, though
    • userbinator 5y ago
      but it’s definitely more than 80 lines of code To this old demoscener, that sounds like a challenge...
  • silisili 5y ago
    I normally don't like comments like this but I think in this case your point stands. DNS is not 'easy' in that you have troves of RFCs and undocumented but expected behaviors to follow, etc. miekg/dns does all of the heavy lifting here. Writing a resolver in it is a fun project, but bragging about the line count is silly.
    • wanderer_ 5y ago
      I think the line count was just to emphasize simplicity and accessibility, rather than to brag.
    • djur 5y ago
      I didn't read the article as "bragging" about the line count, but saying it was short enough to be read casually as a companion to the comic.
    • bragr 5y ago
      Cannot second this enough. miekg/dns is probably tens of thousands of mature lines of code covering all aspects of the DNS protocol. This feels like bragging about creating an minimal http client but really you just used libcurl to do the heavy lifting.
      • djur 5y ago
        Seems more like writing an article where you explain how HTTP caching headers work by using libcurl to write a program to mirror a website. The purpose isn't showing off, it's education.
      • tptacek 5y ago
        It is nothing at all like using libcurl to do an HTTP request, as libcurl handles the semantics of the HTTP protocol, and not just marshaling and unmarshaling HTTP requests and responses. miekg/dns is also 20,000 lines of code most of which are not relevant to the task of doing a recursive lookup.
    • mevdschee 5y ago
      >bragging about the line count is silly. and gatekeeping is silly too..
  • chomp 5y ago
    I smiled at this, though the Sagan quote "If you wish to make an apple pie from scratch, you must first invent the universe" springs to mind. At some point to be reasonable you have to cut off the lower layers of abstraction or else it's madness. Unbound leans on libresolv, so I don't think it's cheating to lean on external libraries - most of the hard part about resolvers is all the crufty RFCs you have to support anyway, not the basic nuts and bolts of resolving an A record.
    • LinuxBender 5y ago
      Somewhat off-topic: Adding song [1] for completeness sake and so that it is stuck in other peoples heads too. [1] - https://amorningfilledwith400billionsuns.ytmnd.com/ https://amorningfilledwith400billionsuns.ytmnd.com/
  • whimsicalism 5y ago
    Agree only because of how the post was titled.
  • dapids 5y ago
    I guess we are calling paths to executables "bash code" now.
  • tptacek 5y ago
    This is a surpassingly facile putdown. `miekg/dns` is being used here as a DNS request/response parser library. There's almost nothing interesting about parsing DNS messages. The interesting logic in the DNS is the recursive lookup, which is the part this post captures. As far as I know (we use the same library for our authority servers) `miekg/dns` doesn't even do recursive lookups. It's not just that, though. It's also that recursive lookups are the probably the most mystifying aspect of DNS. DNS message parsers are straight-line code; there are lots of them, including in POSIX libc. What you don't have in POSIX libc is a function that does a complete from-the-root recursive lookup for a name; most DNS "client" implementations stop at "send the request to the local recursive cache". To see that function implemented in such a tiny amount of code is itself super interesting. Essentially, what you're saying is that the article disappointed you because it purported to be about making pizza, but didn't first specify how to make the oven.
    • BeefWellington 5y ago
      I disagree. The entire client connection is being handled by the imported DNS library: func dnsQuery(name string, server net.IP) *dns.Msg { fmt.Printf("dig -r @%s %s\n", server.String(), name) msg := new(dns.Msg) msg.SetQuestion(name, dns.TypeA) c := new(dns.Client) reply, _, _ := c.Exchange(msg, server.String()+":53") return reply } func main() { name := os.Args[1] if !strings.HasSuffix(name, ".") { name = name + "." } fmt.Println("Result:", resolve(name)) } While I agree it's impressively simply laid out and as an exercise good to see people getting hands-on with underpinning protocols (I'm a fan of this), it also does its job as a follow up to her previous blog post on the subject. People taking umbrage with the title as posted here (and the original site title) are IMO not wrong. The article title has been changed to "A toy DNS resolver" now, so obviously it was contentious enough that she thought to change it.
      • tptacek 5y ago
        What do you think an "entire client connection" is? It's a read and a write from a net.Conn. Read client.go's (c *Client) exchange() (the lowercase one). It's 30 lines of code, and that includes stuff like EDNS0, TSIG, and read deadlines that have nothing to do with the functionality the author is demonstrating. If the author took the time to write a 15 line roundTrip() function that called m.Pack(), udpConn.Write(), udpConn.SetDeadline(1s), and udpConn.Read(), would your argument here evaporate? Then it's not a very good argument, is it?
  • deleted 5y ago
    [deleted]
  • dang 5y ago
    "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    • takeda 5y ago
      I know it starts like a shallow dismissal, but mentioning that code uses another DNS package as a dependency (miekg/dns) has a point. I originally missed that, and I believe many probably too. I don't think author can claim "in 80 lines of code" when the code depends on a much bigger library than itself.
      • comonoid 5y ago
        It's just a generic DNS packet parser, it has no any resolving logic (or it is not used). For learning the resolving process, it is a perfect combo.
    • chlorion 5y ago
      The original title was "Writing a DNS resolver in 80 lines of Go". The comment might seem like its being dismissive with the current title "A toy DNS resolver", but at the time of writing it was not just a shallow dismissal.
      • tptacek 5y ago
        It was a shallow dismissal then too.
• charcircuit 5y ago
  So what happens when there is a cycle in the graph? Does it just DoS the resolver?
  • zamadatix 5y ago
    In this implementation yes, in a full implementation the standards expect the client to handle multiple different scenarios that can result in loops as an error.
  • tptacek 5y ago
    I think a more typically structured recursive resolver is built around a read-through cache structure. Each step of the query asks the cache for more records, which you may or may not already have fetched; if you have, they're just served from the cache.
    • Groxx 5y ago
      That's describing how a cache hit stops the cycle, not how a cache miss doesn't encounter one.
      • tptacek 5y ago
        It's why a cycle doesn't just DOS the server.
        • Groxx 5y ago
          Er. Not with that explanation. The cache won't be hit until a result is found, instead each request produces another request back to through the cycle to try to get that request. Hence DoS.
      • atomicnumber3 5y ago
        The standards tell you specifically to handle not going into infinite resolving loops and such.
• gnabgib 5y ago
  It looks like Julia may have renamed the title? The H1 and html.title are both "A tiny DNS resolver in Go". Little less baited with the new title. The slug contains the old title, not suggesting it was an editorial by OP (but could perhaps do with a similar update on HN?)
  • deleted 5y ago
    [deleted]
• hasmatteam 5y ago
  Go is such a compact pretty language.
  • itake 5y ago
    compact?! - No Safe navigator: require 4 lines of code - No ternary support: require 6 lines of code - No map or filter functions (meaning you have to implement these in like 5-6 lines of code) - Golang relies on built-in generator comments to help alleviate all of the typing.
    • svnpenn 5y ago
      > No Safe navigator: require 4 lines of code > No ternary support: require 6 lines of code > No map or filter functions (meaning you have to implement these in like 5-6 lines of code) I like that Go doesnt have these. Many programmers seem to want these big bloated programming languages. So thats what you end up with, a bunch of slow bloated programming languages. Personally I just want something like C, but with a couple of minimal extra features (maps, built in package management, bounds checking). For me, Go is the closest thing I have found to C, without many of the C pitfalls. > Golang relies on built-in generator comments to help alleviate all of the typing. I have been programming Go for a few years, and I have like one file in all of my code that uses a generator comment. That file is to interface with the Windows API. Regarding pure Go code, I dont use generator comments at all, so I think this comment is overblown.
      • takeda 5y ago
        > I like that Go doesnt have these. Many programmers seem to want these big bloated programming languages. So thats what you end up with, a bunch of slow bloated programming languages. Those features are added to remove bloat from your source code.
        • svnpenn 5y ago
          I don't think you understand what bloat is. Just because an end user program has less lines of code, doesn't mean bloat is reduced. That code gets made up in the API, or the language itself.
• _paulc 5y ago
  Simple python version which does a bit more validation (checks response matches query) and also supports CNAMEs and different record types (37 lines - using dnslib library) - though there is still a lot of logic missing (handling failed NSs, IPv6 only NSs, cycles etc) from dnslib import DNSRecord,DNSError,QTYPE ROOT_NS='198.41.0.4' def find_cname(a,qtype,cname): for r in a.rr: if QTYPE[r.rtype] == qtype and r.rname == cname: return str(r.rdata) elif QTYPE[r.rtype] == 'CNAME' and r.rname == cname: return find_cname(a,qtype,str(r.rdata)) return resolve(str(r.rdata),qtype,ROOT_NS) def resolve(name,qtype='A',ns=ROOT_NS): print(f"dig -r @{ns} {name} {qtype}") q = DNSRecord.question(name,qtype) a = DNSRecord.parse(q.send(ns)) if q.header.id != a.header.id: raise DNSError('Response transaction id does not match query transaction id') for r in a.rr: if QTYPE[r.rtype] == qtype and r.rname == name: return str(r.rdata) elif QTYPE[r.rtype] == 'CNAME' and r.rname == name: return find_cname(a,qtype,str(r.rdata)) for r in a.ar: if QTYPE[r.rtype] == 'A': return resolve(name,qtype,str(r.rdata)) for r in a.auth: if QTYPE[r.rtype] == 'NS': return resolve(name,qtype,resolve(str(r.rdata),'A',ROOT_NS)) raise ValueError("Cant resolve domain") if __name__ == '__main__': import sys def get_arg(i,d): try: return sys.argv[i] except IndexError: return d print(resolve(get_arg(1,"google.com"), get_arg(2,"A"), get_arg(3,ROOT_NS))) Disclaimer: I am author of dnslib [https://github.com/paulc/dnslib https://github.com/paulc/dnslib] Edit: Better CNAME support
  • deleted 5y ago
    [deleted]
• memorable 5y ago
  Just see this one while searching what a DNS resolver is.
• mxjay 5y ago
  Hi, "creeting" may be a typo
• 1vuio0pswjnm7 5y ago
  Many years ago I wrote a "resolver" using only 1. a custom filter program that parses domain name from stdin for its TLD and sends a query the appropriate registry nameserver to get the authoritative nameserver for the domain, and 2. a series of tiny shell scripts, each about 385 characters or less. There are about 10 scripts in total. Each does a specific task. What was interesting is that, for names not already in held in a cache, this system was able to beat the speed of full-blown DNS resolvers like BIND, dnscache, pdns_recursor, unbound, etc. The reason it worked is that DNS resolution tends to follow certain patterns. There were 43 different paths DNS lookups take in the real world that I discovered. Some of them are extremely common. Some of them are extremely rare. These paths are simply the reflection of how people configure DNS. The most common pattern is also the most efficient. It only requires three queries. The least common patterns require many more queries. Arguably some of them could be called evidence of "DNS misconfiguration". The program "brute forces" resolution by trying the most common patterns first. If it succeeds, it exits. If it fails, it will retry using the next most common pattern. When the size growth of the root.zone exploded thanks to new gTLDs, this made the size of the custom filter that parses the TLD from stdin ridiculously large. That said, some of those new TLDs are rare to see and some are not even in active use. Is it even worth including them. The same question arises with all the different patterns a DNS lookup can follow. Some are so rare and so inefficient, why bother "supporting" them when writing a resolver. The problem with DNS resolution IMO is that people believe there should be almost no rules in how someone configures DNS. If a website operator wants to force a user to make 18 DNS queries to get the IP addresss of their website, when it could easily be done in 3-5 queries, she can do that, and people expect a DNS resolver to be able to deal with that scenario. From a user perspective, this is stupid. Obviously these scenarios make for slower lookups. As the number of dependencies increases, i.e., number of different nameservers involved, the likelihood of failure increases. Not to mention this makes writing a DNS resolver more complicated. From comments I read on HN, it sometimes seems that some programmers yearn for complexity. Simple, linear things that work fast and reliably are not interesting. Lord only knows how much we all suffer as a result of that perversion. One HN commenter called it a complexity fetish.
  • 1vuio0pswjnm7 5y ago
    The trick to beating the caches was in part that the filter program had the IP addresses of all the TLD nameservers hardcoded into it. That was one less query I had to make. A cold cache that did not have the TLD nameserver address would have to make that query. I used to worry about the size of this filter program but when I see how people routinely use double-digit MB binaries today, including some very large Go ones for DNS stuff, I guess I could have carried on and ignored it. I could have hardcoded the full spectrum of new gTLDs. It would probably be no larger than the large binaries people use today. The filter program was generated directly from the root.zone file. I think root.zone only changed twice in the eight years I was using this method. Both times it was some rarely used ccTLD replacing a nameserver. Regenerating the filter program, i.e., compiling it, took about 60 seconds. At the time, I was working on another filter that hardcoded the addresses for the "most common" registrar nameservers. Then people started adopting DoT and DoH, I lost interest in unencrypted authoritative DNS^1 and stopped using DNS altogether. Now I store selected bulk DNS data in the memory of a TLS forward proxy. I eliminated the variable delay of DNS when making HTTP requests and I got network-wide "ad blocking" as a side benefit. 1. I awlays thought about starting a registrar that offers encrypted authoritative DNS via DNSCurve, using the CurveDNS forwarder. Why. Because no one else ever did it. I use CurveDNS in the "home lab" and it works flawlessly.
• eggsome 5y ago
  For anyone who want's something tiny, complete and secure check out djbdns. I use it for a 700 user org as a secondary DNS. Never goes above single digit CPU usage (on an AWS nano instance) and currently has 834 days of uptime. One of my favorite pieces of software.
  • longcommonname 5y ago
    Its impressive that it's reliable. It's worrying that your uptime indicates you don't patch frequently.
    • eggsome 5y ago
      As a general rule you make a good point, but in the case of djbdns there has been no need to patch for years. See here: https://cr.yp.to/djbdns/guarantee.html https://cr.yp.to/djbdns/guarantee.html
      • catlifeonmars 5y ago
        Kernel updates?
        • tlamponi 5y ago
          FWIW, there are mechanisms to live-patch the kernel, there were some out of tree approaches (ksplice, kGraft, kpatch) but nowadays, the kernel has native support that those (kGraft and kpatch IIRC) now can use: https://www.kernel.org/doc/html/latest/livepatch/livepatch.html https://www.kernel.org/doc/html/latest/livepatch/livepatch.h... https://wiki.archlinux.org/title/Kernel_live_patching https://wiki.archlinux.org/title/Kernel_live_patching Most enterprise distros provide a service for that, as the actual work is to create the binary patch fixing the security issues at hand, as one can not always just use the upstream version, e.g., if that introduces internal ABI changes or changes locking (order) - as then you'd need to patch X sites atomically at once to ensure nothing falls apart, can be done but hard to get right. So yes, if you're willing to put in the money or work you can have systems that run for years and still are just as secure as those that frequently reboot into new updated kernels.
• pvsukale3 5y ago
  Unrelated: I read the title of this link and I knew who wrote it before even checking the website. Love Julia's writing. Always something from scratch and explained really well.
  • kzzzznot 5y ago
    Julia Evans’ writing is always a joy to read. She’s an excellent speaker too