8 ms·
RFC8482 – Saying Goodbye to ANY
- jedisct1 8y agoBut... Qmail! :)
- wereHamster 8y agoWhat does qmail use it for?
- majke 8y agoQmail used to do ANY. Qmail, as opposed to other software, was sane enough to understand that lack of MX in ANY didn't mean it wasn't there. It meant it wasn't in a cache, so it would retry with just MX. Our initial proposal (to do REFUSED for ANY) indeed had a chance to break some older qmail installations. This is why we engaged in a longer process and found acceptable solution - HINFO. HINFO is both backwards compatible (qmail will work fine) and solves our problems with ANY. Win-win.
- bluejekyll 8y agoTechnically, there can be multiple queries in a single DNS request, though this is under-specified in the RFCs. Also, most authorities don't support it.
- eridius 8y agoWhy did it do ANY?
- msbarnett 8y ago> This is really just an old workaround that's no-longer needed. Let me explain... > Once upon a time, back in 1996, there was a really unfortunate bug in the most popular DNS server software (BIND 4.9.3): it did not respond correctly to "CNAME" requests (that is to say, requests for any CNAME data about a particular domain name). This is critical information that an email server needs to know to do its job. Thankfully, there was a way to work around the problem: "ANY" requests. These requests ask the DNS server, essentially, for any and ALL information it has about the domain name in question, including CNAME information. > These ANY queries have two big problems: > As you might imagine, for big domains with lots of mirrors (e.g. gmail.com), that's a lot of information, and so the response can be quite big. Big responses pose two problems: first, it's a waste of bandwidth, and second, it can expose a bug in qmail's handling of large DNS responses (see the next patch). > ANY queries are often not cached by relaying DNS proxies (for whatever reason), and so ANY queries cause more traffic EVEN behind a caching DNS proxy server.
- zamadatix 8y agoAlso there used to be a bug with CNAME lookups and ANY was a workaround. http://www.memoryhole.net/qmail/#any-to-cname http://www.memoryhole.net/qmail/#any-to-cname
- rietta 8y agoI used to use Qmail! But not since about 2005 when I finally converted to Postfix if memory serves.
- JdeBP 8y agoPatched not to do this for many years, now. By me. * http://jdebp.eu./Softwares/djbwares/qmail-patches.html#any-to-cname http://jdebp.eu./Softwares/djbwares/qmail-patches.html#any-t...
- superkuh 8y ago> This got us thinking: do we really need it? "ANY" is not a popular query type - no legitimate software uses it (with the notable exception of qmail). Cloudflare once again pushes for broad changes to the way the net operates that benefits themselves and other centralized corporate players without any benefit to individuals that actually use the net in the sense of using the internet and not just a web browser. The kind of exploration and direct learning that was possible when I was a kid growing up 90s/00s is slowly being phased out as the money seeps in.
- zamadatix 8y agoWhat exactly about the deprecation of ANY do you disagree with (other than Cloudflare proposed it and it used to exist)? Also the blog post and RFC seem to make convincing arguments for the general health of the internet, do you disagree with these and if so why?
- superkuh 8y agoI've always used it to explore the internet for fun as a person without any profit motive. Cloudflare's argument's are all based around ANY making it slightly harder for them to make a profit. A small chance of DNS amplification attacks in DNS is what Cloudflare thinks about because it's their business. But there's no reason to believe this is more important than individual users wishing to see what servers are behind a domain. Combined with GDPR killing off whois the internet is a much more boring, less transparent place.
- j0057 8y agoLimiting ANY queries to just TCP would have been a reasonable middle ground, but now Cloudflare has a cleaner codebase and does not have to do any difficult thinking around what to do with synthetic rrsets in response to ANY queries. Good for Cloudflare!
- majke 8y ago> Limiting ANY queries to just TCP would have been a reasonable middle This was proposed solution, the problem is that in case of attack, against a valid Authoritative service, launched via open resolvers, the open resolvers would just download Gigabits of ANY traffic with TCP. Read about this here: https://fanf.livejournal.com/140566.html https://fanf.livejournal.com/140566.html
- jdashg 8y agoThis is great. It's so important to try to push for improvements externally, rather than just sucking it up and implementing onerous workarounds. It doesn't always work out, but this sounds like a good and reasonable outcome this time. (Though it's their blog, so of course it sounds that way!)
- anyfoo 8y ago(Title of the submission should probably capitalize “Any” as “ANY”.) I liked the ANY type as a convenience and debugging tool, but I guess the disadvantages far outweigh that perceived convenience. And now that I read about the actual semantics of ANY with respect to caches, turns out I was using it wrong anyway!
- fanf2 8y agoIn recent versions of BIND ANY queries still work pretty well as a debugging tool when `minimal-any` is turned on, because you get the traditional full-fat response over TCP, and dig automatically uses TCP for ANY queries. I implemented `minimal-any` in BIND to reduce problems caused by large responses in DDoS attacks. It helps in situations when RRL is not quite enough.
- anyfoo 8y agoAwesome, thanks a lot for that! I was indeed mostly using ANY through dig (as do most, I suspect), glad to know there's still a way to make it work!
- JdeBP 8y agoMany people were. I and others have repeatedly pointed out that "ANY is not ALL" over the decades. As a randomly selected example, here is Paul Jarc back in 2004: * https://dns.cr.yp.narkive.com/MSBPuJCn/newbie-question-about-recursive-lookups-with-dnscache#post2 https://dns.cr.yp.narkive.com/MSBPuJCn/newbie-question-about...
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- derefr 8y agoEven though I know how it works, and why it works, I'm still kind of surprised that individual DNS records have individual TTLs. It feels unfamiliar/alien, compared to all the other hierarchy-of-caches systems I work with. Every intuition I have about caching would lead me to imagine an alternative implementation of DNS where zones are documents, and intermediate resolvers retrieve and cache entire zone documents from their canonical hosts, in order to respond to queries for individual records from stub resolvers. Is there some big use-case for the DNS records within a zone to have different TTLs? Would it really be so inefficient if the zone as a whole just had one TTL, which was set as the minimum of the TTLs required by any of its records? Does DNS not have spacial locality to its access patterns? I.e., if an intermediate resolver retrieves the A records for a domain, does that not imply that it'll soon also want the MX records for that domain, and the TXT SPF records, and—essentially—all the records with some probability? Is there no justification for pre-fetching these? And, if both of those statements were to be true—that DNS could get by just fine with one TTL per zone, and that DNS queries do have a spacially-local access pattern—then why would you want to design DNS in a way where intermediate resolvers retrieve individual records, rather than entire zone documents? Would a modern ground-up DNS architecture do things this way? (I have the suspicion that the answer lies in security-through-obscurity, vis. the reason AXFRs are prohibited. So, as an addendum: I ask the same questions as above, but this time presuming your DNS daemon has Row-Based Access Control (RBAC), such that an intermediate resolver retrieving a zone document would only see the records its upstream resolver thinks that particular client should be allowed to see—e.g. enterprise Intranet clients get everything, while public Internet clients get a trimmed view. This implies secure DNS, but just take that as a given.)
- simcop2387 8y agoThe problem with this is that there may not be a single zone file. Either because of location aware resolvers, or because the entry won't exist until it's requested. This means that caching the whole zone wouldn't allow for dynamic resolving and would require lots more caches at whatever granularity someone else wants for their local server architecture.
- jcrites 8y agoRecord-level TTLs are useful in practice. Let me give an example. Let's say I am about to make a significant change to the operation of a system. I'm going to flip service.example.com from pointing one place, to pointing somewhere else. If something goes wrong during this deployment, then I want to be able to roll back quickly. In preparation for this deployment, I might lower the TTL on service.example.com considerably. This way I can roll out my change quickly and observe the effect across all clients, and I can also roll it back quickly. Once the deployment is done, and I don't expect to make further changes to service.example.com, then I can raise the TTL again. More generally, you might want the ability to propagate changes to the definition of some names much faster than others, and record-level TTLs let you do this. I might be planning to change service.example.com shortly while not planning to change www.example.com any time soon. The downside of low TTLs is a higher volume of queries and thus higher costs, with some implications for availability. Record TTLs are something that I think about, and consider changing, whenever I'm about to make a significant change to a name. (Yes, if TTLs only existed for zones, then you could still break out service.example.com into its own zone, and define the TTL there. But if you frequently want different TTLs for different records, then having to do this would be inconvenience without benefit. Service.example.com would be part of its parent zone initially, then you'd have to separate it out into its own zone just for the sake of a different TTL, and then merge it back in later. Under this use-case, zone-wide TTLs only will add complexity.)
- drywater 8y agoI’m not convinced that ANY is that bad. They wrote some bad code, they have special goals and all of a sudden ANY is not good for you.
- jlv2 8y agoAgreed!
- sneakernets 8y agoThe problem is that no one could tell reliably what ANY even was. Chalk it up to English being ambiguous in every way imaginable. Or is it "any way" imaginable?
- Someone1234 8y agoPeople are using it for reflected DDoS attacks. This has resulted in DNS providers scoping ANY to the point of breaking it. So even without removing it, it was horribly broken and returned inconsistent results. In essence all this change does is remove the fiction of ANY, with or without RFC8482 ANY wasn't reliable enough for real usage.
- deathanatos 8y agoGiven how bulky ANY is, and the fact that we're changing the standard anyway, would not just restricting ANY to TCP queries "fix" the DDoS issue? The attempted TCP connection would not be formed (the victim would RST it). (I agree with other issues pointed out by the article, and there are other reasons why, as a RR type, I would still axe ANY. But the functionality of being able to query all RRs on a server is often useful for debugging, though I think there are other practical ways to work around that. (Issue a query for many common RR types.))
- deleted 8y ago[deleted]
- kazinator 8y agoWhy would it be harder to implement then simply iterating over the known types and catenating the results of those queries. Say we have A, CNAME and MX type records in our system. If we see ANY, let's convert it up-front into into three queries: A, CNAME and MX. Combine the results and that's it. ELIA5
- anyfoo 8y agoThe article gives some insight. For example, to quote: Should it forward the "ANY" query to authoritative? Should it respond with any record that is already in cache? Should it do some a mixture of the above behaviors? Should it cache the result of "ANY" query and re-use the data for other queries? Pair that with inconsistent implemented behavior.
- kazinator 8y agoIf we treat ANY as universally quantified, for absolute correctness/up-to-dateness we must always send the request upstream, because we are not the authority on all that may exist. However, we can sacrifice a bit of correctness with some reasonable heuristics. If we execute an ANY query for a given domain, we can cache the fact that we did such a query, in addition to caching the records that come back. Then, for some time after that, we may treat those cached records as being everything that is known about that domain; if another ANY query comes in for that domain, we give those records to the client. Furthermore, in an independent cache entry, we can associate the list of record types with that domain: it informs us that for that domain, records of those types are known, or have been known. When we get an ANY query for that domain, if we have this list, we can walk the list of types and query upstream for the missing records that have expired from our cache, rather than forwarding the ANY request. Then we serve the client a mixture of cached and fresh records.
- anyfoo 8y agoOh my. Multiple heterogenous but interacting caches? Heuristics? And if I understood it correctly, if any of the records you got from ANY are expired, you selectively query for new records from a list of cached types, not using ANY? What if the upstream added a record from another type in the meantime? Why have ANY at all then? How is any of that "simple"? How do you transition the existing infrastructure to that behavior? Why?
- mehrdadn 8y agoCould someone explain why standards are "RFC"s? Can't anyone request comments for something? Why would a request for comments be treated as a standard?
- pas 8y agonot all RFCs are standards. RFCs that become one are labeled as STD and a number, but nobody uses those (afaik) https://www.ietf.org/rfc/std-index.txt https://www.ietf.org/rfc/std-index.txt https://en.m.wikipedia.org/wiki/Internet_Standard https://en.m.wikipedia.org/wiki/Internet_Standard
- deleted 8y ago[deleted]
- wnissen 8y agoBecause the internet used to be unimaginably small. It really was just a couple of researchers talking to each other. They weren't co-located (the entire point of internetworking) so they had to mail stuff to each other, but it was more like when you send a copy of a document to your co-worker down the hall. "Hey, Jane, watcha think about this?" The first telnet (predecessor to SSH) RFC was 9 pages, including an example of the connection handshake.
- hak8or 8y agoYears ago when the internet was still a bunch of universities, the main few folks working in the network were often times tuning hardware like routers. I do not remember what the first rfc was about, but it was genuinely a "request for comments" document mailed it to others. Ever since then it stuck.
- ZWoz 8y agoIf you are interested history behind RFCs and early Internet, I suggest reading "Where wizards stay up late: the origins of the Internet" from Katie Hafner and Matthew Lyon. Thats good book about arpanet formation and how arpanet and other networks together became Internet.
- dboreham 8y agoHoly crap I remember when RFCs had three digits!
- LinuxBender 8y agoI am happy to see it go. For years I've had to block it in iptables using: iptables -t raw -I PREROUTING -i eth0 -p udp -m udp --dport 53 -m string --hex-string "|0000ff0001|" --algo bm --from 40 --to 65535 -j DROP Most bots are poorly coded, so most of them don't try the individual records when ANY times out. ANY is also a decent generalized amplification attack vector. There are plenty of other ways to do amplification attacks, but this one is quite brain dead. Thankyou to the folks at CloudFlare for submitting this.