3 ms·
Many years ago I wrote a "resolver" using only 1. a custom filter program that parses domain name from stdin for its TLD and sends a query the appropriate regis
by 1vuio0pswjnm7 5y ago
Many years ago I wrote a "resolver" using only 1. a custom filter program that parses domain name from stdin for its TLD and sends a query the appropriate registry nameserver to get the authoritative nameserver for the domain, and 2. a series of tiny shell scripts, each about 385 characters or less. There are about 10 scripts in total. Each does a specific task.
What was interesting is that, for names not already in held in a cache, this system was able to beat the speed of full-blown DNS resolvers like BIND, dnscache, pdns_recursor, unbound, etc.
The reason it worked is that DNS resolution tends to follow certain patterns. There were 43 different paths DNS lookups take in the real world that I discovered. Some of them are extremely common. Some of them are extremely rare. These paths are simply the reflection of how people configure DNS. The most common pattern is also the most efficient. It only requires three queries. The least common patterns require many more queries. Arguably some of them could be called evidence of "DNS misconfiguration".
The program "brute forces" resolution by trying the most common patterns first. If it succeeds, it exits. If it fails, it will retry using the next most common pattern.
When the size growth of the root.zone exploded thanks to new gTLDs, this made the size of the custom filter that parses the TLD from stdin ridiculously large. That said, some of those new TLDs are rare to see and some are not even in active use. Is it even worth including them. The same question arises with all the different patterns a DNS lookup can follow. Some are so rare and so inefficient, why bother "supporting" them when writing a resolver.
The problem with DNS resolution IMO is that people believe there should be almost no rules in how someone configures DNS. If a website operator wants to force a user to make 18 DNS queries to get the IP addresss of their website, when it could easily be done in 3-5 queries, she can do that, and people expect a DNS resolver to be able to deal with that scenario. From a user perspective, this is stupid. Obviously these scenarios make for slower lookups. As the number of dependencies increases, i.e., number of different nameservers involved, the likelihood of failure increases. Not to mention this makes writing a DNS resolver more complicated.
From comments I read on HN, it sometimes seems that some programmers yearn for complexity. Simple, linear things that work fast and reliably are not interesting. Lord only knows how much we all suffer as a result of that perversion. One HN commenter called it a complexity fetish.
- 1vuio0pswjnm7 5y agoThe trick to beating the caches was in part that the filter program had the IP addresses of all the TLD nameservers hardcoded into it. That was one less query I had to make. A cold cache that did not have the TLD nameserver address would have to make that query. I used to worry about the size of this filter program but when I see how people routinely use double-digit MB binaries today, including some very large Go ones for DNS stuff, I guess I could have carried on and ignored it. I could have hardcoded the full spectrum of new gTLDs. It would probably be no larger than the large binaries people use today. The filter program was generated directly from the root.zone file. I think root.zone only changed twice in the eight years I was using this method. Both times it was some rarely used ccTLD replacing a nameserver. Regenerating the filter program, i.e., compiling it, took about 60 seconds. At the time, I was working on another filter that hardcoded the addresses for the "most common" registrar nameservers. Then people started adopting DoT and DoH, I lost interest in unencrypted authoritative DNS^1 and stopped using DNS altogether. Now I store selected bulk DNS data in the memory of a TLS forward proxy. I eliminated the variable delay of DNS when making HTTP requests and I got network-wide "ad blocking" as a side benefit. 1. I awlays thought about starting a registrar that offers encrypted authoritative DNS via DNSCurve, using the CurveDNS forwarder. Why. Because no one else ever did it. I use CurveDNS in the "home lab" and it works flawlessly.