7 ms·
Scaling the Root of the DNS
- vlovich123 6y agoI wonder if the only change that needs to happen is that applications driving a significant amount of traffic (i.e. Chrome) & DNS owners of other domains need to start bearing the costs here. Then we wouldn't be dealing with cost externalities & tragedy of the commons. On the other hand, introduce revenue streams & you start focusing on optimizing the revenue stream instead of solving the problem. I wonder if there's a different way to make the costs felt that isn't directly revenue to align incentives. Like slow down or fail queries for zones who's authoritative servers don't implement the features needed to keep scaling under control. Then make sure that there's revenue to the root servers from non-root servers based on load so that costs are tracked & managed.
- temp667 6y agoJust let google run some of the roots - I'm serious. For all the whining about bandwidth and traffic, this issue is so small on the scale of google (which includes youtube) and netflix traffic flows it's hard to even imagine. Their secondary resolvers run at huge volumes, everything from 8.8.8.8 to the dns for their mail and other web properties.
- jeffbee 6y agoThe problem is it does not help to have one or two fast roots. The root server model only really works when the roots each have similar capacity.
- temp667 6y agoA root is not one static machine. Google can serve 1/3 of traffic for all routes, AWS could do the same. I don't think folks realize how SMALL the root dns traffic is. Youtube / netflix / amazon prime video are just orders and orders of magnitude more demanding in terms of bandwidth - every second they are pumping MULTIPLE HD streams to millions of folks. They measure bandwidth terrabits per second. Even at 30 frames per second, we are talking incredible bandwidth. DNS responses should be 1K or less (often .5K). About 40 netflix viewers at their top quality use the same bandwidth? Netflix has far more than 40 users.
- jeffbee 6y agoYou think the root operators should just delegate part of the traffic of all letters' VIPs to google and amazon? It's not crazy, but it would mean the entire root would have a shared-fate failure mode.
- tremon 6y agoI don't understand. What's the failure mode you're thinking of?
- jeffbee 6y agoSomeone at Google pushes yet another invalid config of death, taking out their global network and, with this proposal, also taking down 1/3rd of all DNS root capacity.
- temp667 6y agoYou could easily 3x provision root dns (or more). I'm just pointing out that for all the complaining about google efforts to block domain jacking (which was endemic before they started their efforts) scaling the root is doable, and if the existing maintainers are too incompetent to scale it then maybe others should be allowed to. I mean, how do they handle denial of service attacks which are going to be orders of magnitude higher in volume? The logic here is so warped. https://www.digitalinformationworld.com/2020/08/a-major-bug-in-google-chrome-is-causing-a-massive-load-on-global-root-dns-servers.html https://www.digitalinformationworld.com/2020/08/a-major-bug-... Rather than the solution being to allow for domain hijacking by turning off the redirect detectore, how about verisign, which kicked this whole thing off recently AND way back with their "sitefinder" garbage get a clue and either propose another solution to domain jacking or scale the root a bit.
- DenseComet 6y agoThere already is precedence for this. Cloudflare provides a huge chunk of F-root servers. https://blog.cloudflare.com/f-root/ https://blog.cloudflare.com/f-root/
- JdeBP 6y agoYou've conflated the proxy DNS servers, that do grunt work of query resolution (or forward to other proxies that do), with the content DNS servers. It is not the content DNS servers that are causing the scaling problems, it is the proxy DNS servers, and the DNS clients and applications softwares that drive them. * http://jdebp.uk./FGA/dns-server-roles.html http://jdebp.uk./FGA/dns-server-roles.html The change that needs to happen is the thing that most people nowadays tell one first, that this paper in contrast leaves all of the way to the end: run your own content DNS server(s) for the root, on your own LAN or Intranet. It's fairly simple to do, and it's an idea that has been around for over two decades, now. * https://news.ycombinator.com/item?id=24277855 https://news.ycombinator.com/item?id=24277855 * https://news.ycombinator.com/item?id=19526108 https://news.ycombinator.com/item?id=19526108 * http://cr.yp.to/dnsroot.html http://cr.yp.to/dnsroot.html (dates back to 2001) * https://news.ycombinator.com/item?id=24280536 https://news.ycombinator.com/item?id=24280536 et al. It takes a long time in the DNS world for such grassroots ideas to percolate to the level that they get circulated by corporate officers. This article is, for example, still propounding that every resolving proxy DNS server works like ISC's BIND, and has a "priming query". This is untrue in reality, and is one of the differences between BIND and some of the softwares (e.g. dnscache and MaraDNS) that came along to replace it a couple of decades ago. * https://maradns.samiam.org/faq.html#roothints https://maradns.samiam.org/faq.html#roothints Similarly the argument that Google should use a subdomain, presented in this article without any counterpoint at all, has long since been countered by pointing out that this makes the malicious actors' lives easier; as conditional forwarding and split-horizon DNS service, which are well-known things, would then make it easier to fool Google Chrome. And there have been other counterarguments made, years ago. * https://news.ycombinator.com/item?id=3609498 https://news.ycombinator.com/item?id=3609498 (yes, 2012)
- dkdk8283 6y agoI think your terms are incorrect. Proxy queries in your context are referred to as recursive and content is authoritative.
- JdeBP 6y agoThese aren't types of query, they are types of server. viovich123 was talking about content DNS servers that cause the scaling problems, and penalizing the people who run them. But it is the proxy DNS servers, and the DNS clients and applications softwares that drive them, that are the cause here. The terms are correct, and are derived from HTTP. You aren't grasping them correctly.
- jeffbee 6y agoIt's only 1300 QPS per site. Is just plain old better programming on the table? An ordinary PC should be capable of barfing out NXDOMIAN responses at, conservatively, 100000 per second.
- temp667 6y agoI'm looking into this a bit more for some basic math. It looks like QPS is around 1.3M per second? Am I way off? What the bandwidth? I'm getting 1GBps or less which seems pretty low? If the root can't handle this how does it handle DOS floods even? Should we just turn some of the root over to some of the bigger players that can actually scale (AWS / GCP / Azure / Netflix etc)? These rates seem so small for this level of complaining about google et al.
- temp667 6y agoDigging a bit more it looks like some of just the public DNS servers are doing WAY more traffic than this. Google was doing 810K qps on 8.8.8.8 alone in 2012 (8 YEARs ago). Can't they offer to operate some of the root zone then if it's so hard for these other folks to manage it (maybe do something with AWS / Azure etc).
- jhalstead 6y agoAs of the first graph in this August 2018 blog post [0], it seems like Google's Public DNS was handling about 14-18M QPS (1.2-1.6T per day). [0] https://security.googleblog.com/2018/08/google-public-dns-turns-8888-years-old.html?m=1 https://security.googleblog.com/2018/08/google-public-dns-tu...
- geofft 6y agoYeah, the conclusion I'm getting is, how do I sign up to be a root? Chrome's NXDOMAIN detection is good for the world and it sounds like handling 50% of root server traffic would not be that hard.
- tinus_hn 6y agoIf you install a service like pi-hole, does that query the root servers?
- monkpit 6y agoThis could be much more direct. 30 paragraphs in and I don’t really see anything of substance.
- chaz6 6y agoHow about a system for root zone distribution akin to rpki using bittorrent where updates are signed by a trusted key? Then any resolver could easily be a root server.
- akvadrako 6y agoThe root zone is already signed and there is a distribution network in place – that's how caching DNS servers work. It's already very scalable.
- rasengan 6y agoMoving the DNS root to blockchain is one of the best use cases for blockchain [1] given the numerous stakeholders (name owners) and type of information required to be in consensus. This instantly serves to distribute DNS root queries, instantly scaling the root. [1] https://handshake.org/ https://handshake.org/
- CharlesW 6y agoIf one of the best use cases for blockchain is "data about name owners" and "DNS information", the future of blockchain is bleaker than I'd imagined. Handshake isn't a super-persuasive example to cite, since it's backed by a minor altcoin that's primary use case (according to CoinDesk) is crypto day-trading.[1][2] [1] "Update (May 28, 20:11 UTC): The headline and body of this article have been edited to clarify there were reportedly $10 million worth of HNS trades on Namebase since the token launched, but* only a fraction of those users bought domain names." https://www.coindesk.com/handshake-domains-bring-in-10m-as-race-for-censorship-resistant-websites-heats-up https://www.coindesk.com/handshake-domains-bring-in-10m-as-r... [2] https://davidgerard.co.uk/blockchain/2020/05/31/news-not-craig-wrights-bitcoins-how-tether-flows-goldman-sachs-spurns-bitcoin-calibra-is-now-novi/ https://davidgerard.co.uk/blockchain/2020/05/31/news-not-cra...
- rasengan 6y agoThat article likely didn't review the activity on chain [1]. [1] https://dns.live/ https://dns.live/
- throw0101a 6y agoChrome (and derivatives) need to start behaving better: > The motivation for the Google’s Chrome browser to do this is pretty obvious. Many ISPs perform NXDOMAIN substitution in their DNS resolvers and replace the “no such domain” response with a synthetic pointer to their own search page, which allows them to monetize all those No such domain” user typos. From Google's perspective this NXDOMAIN substitution is not well regarded. Search is a major path to advertisement placement and advertising is Google's core revenue. So how can Google's Chrome browser platform detect network environments where NXDOMAIN substitution is happening? Simple. Test the DNS with its own nonce random string queries to see if NXDOMAIN substitution is taking place. > This supposedly innocuous probe of the DNS should’ve been pretty harmless. But there is a lot of Chrome out there these days. Some two thirds of all browser use in today’s Internet is reported to use the Chrome browser, and the number rises when you include products such as Edge that are based on the Chrome engine. Consequently, it is perhaps unsurprising to learn that according to this report these probes are now taking up some 50% of the total root server traffic (Figure 3). I understand that hijacking NXDOMAIN is bad, but Google is now 'punishing' the root DNS operators with extra traffic when the actual perps are the bad-actor ISPs. A bit of collateral damage there. And would it be so bad for Google/Chrome to simply let the NXDOMAIN hijacking slide? They seem to be externalizing costs to help their business model.
- remus 6y ago> And would it be so bad for Google/Chrome to simply let the NXDOMAIN hijacking slide? Yes, it would. It's a workaround to a problem created by shitty ISPs, go after the ISPs if you want to solve the problem.
- throw0101a 6y agoGreat. And what is your idea for to "go after the ISPs"? What does 'going after' them entail? And as someone with a non-shitty ISP, why should I be part of the problem of excess traffic to the root servers?
- JdeBP 6y agoYou need to know a bit of history, namely that one of the earliest "bad actors" here was Verisign, the operator of the com. and net. registry, and of the "A" and "J" ICANN root content servers. * http://jdebp.uk./FGA/verisign-internet-coup.html http://jdebp.uk./FGA/verisign-internet-coup.html
- throwawaysea 6y agoMy knowledge is limited here but at some level there is a limit to the amount of traffic a single endpoint can handle right? For example if something is distributing queries in a round robin manner, isn’t there a limit on how much load that can ultimately handle and distribute to other computers? What is that limit and what constrains it?
- wiml 6y agoIf only there were some technology that could verify whether a DNS query/response had been spoofed, without interfering with caching and proxying, and without the privacy and centralization-of-control problems of DoH. If only such a technology had already been widely deployed in the dns root and many caches and resolvers.
- NKosmatos 6y agoOut of curiosity, how big (in GBs/TBs) are the root files? If I setup a local DNS server (Raspberry or something more powerful) could I duplicate or cache the root file somehow? IMHO the various ISPs should contribute to the cost and technical needs of the root servers or even better utilize blockchain, torrent or other distributed network concepts.
- vesinisa 6y agoThe root zone has grown considerably thanks to a number of new gTLDs, but currently sits at around 2 MB: https://www.iana.org/domains/root/files https://www.iana.org/domains/root/files That's in human-readable ASCII. A wire format could probably shave a chunck off that, but distributing deltas instead of re-downloading the whole file each time a change is made would further reduce the required data transfer amounts to mere bytes per day on average.
- NKosmatos 6y agoThanks for your reply, I was thinking about the actual files containing the DNS entries and not the root file which contains all of the DNS servers. As I understand it the root file is needed in case you want to bootstrap your own DNS server so as to be able to find all the other authoritative servers. What about the actual DNS database files?
- vesinisa 6y agoYou are talking about the root seed list (the NS records for the root label .) - essentially the anycast IPv4/6 addresses for the 13 root servers. This file is provided in the first section of the linked page. The second file on the page contains the actual current root zone, which is a 2 MB file containing all the delegation entries for TLDs (like com., org. and de.) immediately below the DNS root (.). This is what the article is talking about caching locally at your ISP's recursive name server. The whole global DNS can never be downloaded or even built as a local database due to its distributed nature, and also for technical reasons. Below the root zone at TLD level (like com.) there is no way to enumerate all the individual registrations (like google.com., ycombinator.com., microsoft.com. etc) You always query for a specific individual name + record pair, instead of being allowed to download the entire zone. So even if you somehow would be able to contact every authoritative DNS server on Internet at some point in time, there is no standard query to ask them to give you all their records, making building a full local copy of the global DNS database impossible.