8 ms·
CNAME Flattening: RFC-Compliant CNAMEs at a Domain's Root
- rlpb 13y ago"The Inflexibility of DNS" This problem would not exist if clients (eg. web browsers) supported SRV record lookups. The SRV fixes a missing abstraction by mapping names that users see (eg. foo.com) to actual server names (eg. webserv42.foo.com). Without SRV record lookups, web browsers must look up the bare domain, and expect to get an address, for this to work. CNAMEs were not designed to be used for this kind of situation; it's a square peg in a round hole. This isn't due to the inflexibility of DNS; it's due to the inflexibility of clients that ignore newer advancements in the protocol. Of course, this does mean that web hosts are stuck.
- teddyh 13y agoThis idea that web browsers are silly and should just start using SRV records gets a lot of traction, and I would like it if it were true. Unfortunately, the SRV specification prohibits anyone from using SRV records for protocols where the protocol’s definition does not specify the use of SRV records. I.e. since HTTP does not specify the use of SRV records, web browsers are not allowed to use SRV lookups to locate HTTP servers. The ones to blame are, instead, HTTP standards writers, which have not written a new HTTP standard taking advantage of the wonderful thing which SRV records are.
- nraynaud 13y agowhat about SPDY (or TLS if it's the relevant layer)?
- sp332 13y agoThat would cause a fragmentation where SPDY traffic might get directed to a completely different server than HTTP traffic. I mean, I know they're served from different ports already but this seem extra brittle.
- ithkuil 13y agothe different port clearly visible since spdy URLs are https (which defaults to 443)
- jessaustin 13y agoTo be fair, the SRV spec, rfc2782, could have been written (or could be updated) to allow other, more flexible methods. Actually, that spec seems to contradict itself, saying first "...for applications where the relevant protocol specification indicates that clients should use the SRV record..." but later on the same page "The symbolic name of the desired service, as defined in Assigned Numbers [STD 2] or locally." At IANA [0] it appears that only three protocols actually use the assigned numbers infrastructure to handle this, but more could, and in the absence of new RFCs clients could adopt a convention for determining "symbolic service names" for non-SRV-aware protocols. [0] http://www.iana.org/protocols http://www.iana.org/protocols
- teddyh 13y ago(Oh, I see what you meant now.) The things on that page are protocols which require further allocation of numbers and/or names by IANA, and is not directly related to what protocols specifies the use of SRV records. The “three protocols” you mention are protocols which require further allocation of numbers and/or names in their use of SRV records in some way. This does not mean that only three protocols specifies the use of SRV records.
- teddyh 13y agoThat’s the wrong IANA link; you want this one: https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml https://www.iana.org/assignments/service-names-port-numbers/... There are not three, but thousands of protocol names registered.
- asdfaoeu 13y agoYou'll note http is specified there.
- teddyh 13y agoThe list I linked to is a list of all protocols known by IANA. Whether the use of SRV records is allowed by the individual standards is not recorded by IANA; one would have to search each individual standard to find out. HTTP, notably, does not specify the use of SRV records.
- Zr40 13y agoRFC2782 does not prohibit the use of SRV records with those protocols. In general, it is expected that SRV records will be used by clients for applications where the relevant protocol specification indicates that clients should use the SRV record. Such specification MUST define the symbolic name to be used in the Service field of the SRV record as described below. It also MUST include security considerations. Service SRV records SHOULD NOT be used in the absence of such specification. RFC2119 defines 'SHOULD NOT' as follows: This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.
- teddyh 13y agoMaybe “prohibit” was the wrong word. On the other hand, I would argue that a “SHOULD NOT” is stronger than you think: The HTTP standard (RFC 2616) says that: “[…] the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered "safe". This allows user agents to represent other methods, such as POST, PUT and DELETE, in a special way, so that the user is made aware of the fact that a possibly unsafe action is being requested. I would argue that very few people would claim that it’s OK for HTTP servers to have HEAD and GET requests be non-idempotent just because they feel like it. “SHOULD NOT” prohibitions should not be ignored just because you really want to do it.
- MichaelGG 13y agoDo you really think that's the reason? Specs should be ignored when they state things beyond their scope. The a restriction on DNS depending on if a newer HTTP spec definitely something that can and should be ignored if it helps clients.
- teddyh 13y agoI really think that a “SHOULD NOT” in an RFC is a sufficient reason to not do a thing, yes. I further think that specifying the usage of SRV record is wholly in scope of the RFC specifying the SRV record.
- jlgaddis 13y agopkgng, FreeBSD's new package management tool, uses SRV records (by default) with HTTP: $ dig +short srv _http._tcp.pkg.freebsd.org 20 10 80 pkg0.isc.freebsd.org. 50 10 80 pkg0.bme.freebsd.org. 10 10 80 pkg0.nyi.freebsd.org. 90 10 80 pkg0.ydx.freebsd.org. Technically, though, it's not being used by/with a web browser. It generated enough confusion (as evidenced by the frequent "how come I can't {connect to|resolve} pkg.freebsd.org?" questions) that there's now an A RR and a page explaining it: http://pkg.freebsd.org/ http://pkg.freebsd.org/
- joshstrange 13y agoThis is good news, I recently had to migrate off of CloudFlare because of issues with CNAME records on my root[0]. I might switch back as my site is static and could benefit from being behind CF. [0] http://joshstrange.com/why-its-a-bad-idea-to-put-a-cname-record-on-your-root-domain/ http://joshstrange.com/why-its-a-bad-idea-to-put-a-cname-rec... HN discussion: https://news.ycombinator.com/item?id=7293512 https://news.ycombinator.com/item?id=7293512 CF CEO commenting about CNAME flattening 28 days ago: https://news.ycombinator.com/item?id=7295215 https://news.ycombinator.com/item?id=7295215
- eik3_de 13y ago> "The supply of IP addresses is limited" s/IP/IPv4/
- Pacabel 13y agoIn practice today, IP basically is just IPv4. And that won't really change until IPv6 is supported by more than a comparatively small handful of networks and users.
- eik3_de 13y agoTrue, but CloudFlare should be a company driving v6 forward, i.e. mentioning that the address problem only exists in v4. Make people aware that v4 is history and present. Sure, we currently have to deal with it, but v6 and v4 should be treated equally. Esp. when talking about problems v6 solves.
- justincormack 13y agoThey are driving ipv6 forward, they will transparently provide AAAA records for sites they host unless you opt out, and also support ipv6 only backends which they will provide ipv4 for.
- nly 13y agoFacebook has fully deployed IPv6[0], with almost all of their internal traffic now IPv6. They actually ran out of private IPv4 space to use. IPv6 is here. If you're setting up a new service today without it you should be shot. [0] https://www.dropbox.com/s/doazzo5ygu3idna/WorldIPv6Congress-IPv6_LH%20v2.pdf https://www.dropbox.com/s/doazzo5ygu3idna/WorldIPv6Congress-...
- michaelmior 13y agoA comment on the article makes a good point that this seems to break resolution of IPs based on geography. Perhaps CloudFlare takes this into account with their caching mechanism, but this is unclear.
- samcrawford 13y agoIt's hard to see how they could account for it perfectly. I suppose they could inject an EDNS client subnet option into the request they make upstream (with the original client's IP here), but that assumes that the target supports this option.
- justincormack 13y agoResolution of IPs based on geography is already fairly broken in many cases. But it should resolve based on where the cloudflare server is, which should be ok.
- michaelmior 13y agoThat's the point. This is arguably worse than resolving based on the location of the client. Although if the server doing the caching is an edge server already located close to the client, this is less of an issue.
- jgrahamc 13y agoCloudFlare does Anycast for both DNS and HTTP so the DNS resolver will be 'close' to the client and the HTTP connection will be made automatically to the 'closest' CloudFlare PoP.
- dfc 13y agoGoogle DNS and OpenDNS also cause some serious problems with geographic results. My go to example is www.netflix.com. I live in NY and Google DNS always points me to us-west instead of us-east. dfc@fob-xray:~$ host www.netflix.com # local unbound dns www.netflix.com is an alias for www.us-east-1.netflix.com. www.us-east-1.netflix.com is an alias for www.us-east-1.prodaa.netflix.com. www.us-east-1.prodaa.netflix.com has address 50.16.245.238 www.us-east-1.prodaa.netflix.com has IPv6 address 2406:da00:ff00::6b14:f251 dfc@fob-xray:~$ host www.netflix.com 8.8.8.8 # 8.8.4.4 Gives same response Using domain server: Name: 8.8.8.8 Address: 8.8.8.8#53 Aliases: www.netflix.com is an alias for www.us-west-2.netflix.com. www.us-west-2.netflix.com is an alias for www.us-west-2.prodaa.netflix.com. www.us-west-2.prodaa.netflix.com has address 54.245.96.95 www.us-west-2.prodaa.netflix.com has IPv6 address 2620:108:700f::3270:5f36
- time4tea 13y agosomewhat similar to dnsmadeeasy ANAME which has been going for years?
- jessaustin 13y agoDo clients have to know how to deal with ANAMEs? If so, this seems more backwards-compatible.
- time4tea 13y agono they just exist on the server, and are mapped by their service, no client change (otherwise it would be a bit pointless...)
- peterwwillis 13y agoANAMEs also seem to handle geolocation. http://www.dnsmadeeasy.com/services/aname-records/ http://www.dnsmadeeasy.com/services/aname-records/ How Does This Improve CDN Services? When combined with the Global Traffic Director service from DNS Made Easy, ANAME records provide an extremely flexible and powerful solution to keep your CDN service running optimally. As soon as an ANAME record is defined with a particular GTD region (other than Default) the ANAME service ensures that it will only use DNS Made Easy’s resolving name servers that are in that particular region. For example, if you create an ANAME record with “Europe” as the GTD region, then only resolving name servers in Europe would be used to create the A records for this new record type. This ensures that your traffic continues to be routed regionally as planned with your worldwide infrastructure. Use of ANAME records are fully compatible with GTD service and optimize use of a CDN provider. This provides the complete power and flexibility that enterprise companies require from their networks today.
- zimbatm 13y agoIt's disappointing that prior art isn't mentioned in the article. DNSMadeEasy's ANAME record does the same. I believe other providers like Route53 also call it an ALIAS record. It's also good to know that GeoDNS resolution won't work as expected since the location of the resolver is going to be the DNS provider and not the client. EDIT: Actually DNSMadeEasy claims that the resolution is done locally in each of their region so the issue is somewhat mitigated.
- michaelbuckbee 13y agoI think it's about split between ALIAS and ANAME at the different providers. DNSMadeEasy -> ANAME DNSimple -> ALIAS EasyDNS -> ANAME PointDNS -> ALIAS
- statico 13y agoAWS Route 53 aliases aren't ANAME/ALIAS -- they appear to only let you point a record at an AWS address, not any arbitrary IP. (Correct me if I'm wrong.)
- colmmacc 13y agoDisclaimer: I work on Route 53. You can ALIAS to record sets within your own zone too, which can contain any arbitrary IP. This is useful when composing routing types (e.g. weighted plus latency). But we don't support ALIASing to record sets hosted on other DNS providers. It is possible to ALIAS to a CloudFront distribution though, and that CloudFront distribution can in turn use any arbitrary name as its origin. It's also possible to ALIAS to an S3 bucket, which can be configured with a redirect to any name of your choice. Remote DNS-level ALIAS is something that we're always considering, and we're interested in feedback. If we support it, it's unlikely our 100% availability SLA will apply, that there may be inter-operability issues with CDNs (though we recently announced our support for edns-client-subnet: http://aws.typepad.com/aws/2014/04/improved-cloudfront-performance-with-edns-client-subnet-support.html http://aws.typepad.com/aws/2014/04/improved-cloudfront-perfo...) and incompatibilities with our future DNSSEC support.
- zimbatm 13y ago
- dfc 13y ago> if you signed up via a hosting partner then you may have gotten a > CNAME that...SNIP...will now resolve directly from your domain to an > IP address. This makes DNS resolution a bit faster and also better > obscures the fact that you're using CloudFlare. To what end are we obscuring the use of cloudflare? And how useful is the "cloudflara obscura" if the responses are being served by the cloudflare machines listed in the NS records? dfc@fob-xray:~$ dig +short ns nasdaqprivatemarket.com jill.ns.cloudflare.com. igor.ns.cloudflare.com. dfc@fob-xray:~$ dig +short ns skyprep.com mark.ns.cloudflare.com. vera.ns.cloudflare.com. dfc@fob-xray:~$ dig +short ns econsultancy.com cody.ns.cloudflare.com. kay.ns.cloudflare.com. dfc@fob-xray:~$ dig +short ns jct.com ben.ns.cloudflare.com. iris.ns.cloudflare.com. I am not trying to be nit picky. I am genuinely curious, DNS is one of those things I find very interesting.
- jlgaddis 13y ago> "... better obscures ..." In their defense, they didn't say it completely* obscures the use of CloudFlare.
- dfc 13y agoThe real question is the first question: What is the point of obscuring? I probably should have skipped the bit about the effectiveness.
- evilDagmar 13y agoAs best I can tell it seems to be a very heavily veiled excuse to get their web service customers to sign up for Cloudflare hosting their DNS zones as well. Traffic balancing and redirection via DNS responses isn't new ground--other people just don't conflate it with CNAME kludgery.
- xenophonf 13y agoThis is a great hack, but whoever explained C pointers to the copywriter did a bad job. Maybe they confused in-process pointers with kernel virtual memory management, but the operating system doesn't reach into a process's address space and move memory blocks around and rewrite pointers. A language runtime or a memory management library can do that, but it's a little more complicated than pointer-to-a-block-that-gets-moved. It's more like using indirect or doubly indirect pointers and stuff. Grokking pointers is hard enough as is. I weep for all the wannabe systems programmers who will get confused by this otherwise very nice announcement.
- dexcs 13y agoOther people had this problem to. Great to see that cloudflare solves such problems. http://translate.google.de/translate?sl=de&tl=en&js=y&prev=_t&hl=de&ie=UTF-8&u=http%3A%2F%2Fwww.eklenet.de%2Farchives%2FMailserver-ignorieren-MX-Eintraege%2C-wenn-ein-CNAME-gesetzt-ist-29.html&edit-text= http://translate.google.de/translate?sl=de&tl=en&js=y&prev=_...
- buro9 13y agoI wondered how this technically works, and the answer is over here: https://support.cloudflare.com/hc/en-us/articles/200169056-CNAME-Flattening-RFC-compliant-support-for-CNAME-at-the-root https://support.cloudflare.com/hc/en-us/articles/200169056-C... CloudFlare uses CNAME Flattening to present a CNAME as an A record. The CloudFlare server generating the response follows the CNAME chain so that the response to the request from the client (for example, the browser) contains only non-CNAME data -- usually, an IP address. This effectively creates an A or an AAAA record. Without flattening, CloudFlare would serve the CNAME record directly to the recursor (the DNS resolver that translates the domain name to an IP address). They just resolve the destination and serve it as an A record.
- jlgaddis 13y ago> I wondered how this technically works, and the answer is over here: It was described in the article but perhaps not as clearly: "What happens is that, if there's a CNAME at the root, rather than returning that record directly we recurse through the CNAME chain ourselves until we find an A Record. At that point, we return the IP address associated with the A Record."
- evilDagmar 13y agoThis doesn't seem very... sane. It does work, but the explanation is wobbly at best. This appears to be a King Kong defense converted into a sales tactic. In their theoretical problem, foo.com needs to be moved to somewhere else in a hurry to reduce an unpleasant load level. No problem--you simply change where foo.com points to. Normal people change the A record and call it a day. Why would you want to involve a CNAME in this? Perhaps if you needed to direct the query to some other server because it's managed by some other entity. But wait... This requires the other entity to serve your DNS records. Not exactly a problem--even though it rather uncomfortably resembles a horizontal referral--so long as we exercise restraint, it only slows down the query a little bit. So now the lookup goes to the server your domain (foo.com) is hosted on, and needs to goes to the other server to get the foo.com.fancynametheycanmove.webprovider.com, except no... that won't really work because your DNS provider hosting foo.com would need to allow you to put CNAMEs at the root of the zone, and CNAMEs are singleton records. The solution? Why it's to use webprovider.com to host your foo.com DNS as well! Then they can do this because their magic DNS server does all those other lookup steps for the querying client. But wait... if they're going to be hosting the foo.com zone, why don't they just change the A record in the first place? Why on /earth/ would it be useful to involve a bunch of CNAME jumps in that? For that matter, how is this even slightly usefully different than a normal global traffic manager issuing different responses for DNS based on load and server pool configuration that for the love of all that is holy I hope they already know about if they're trying to run a "cloud provider". The reason we don't have CNAMEs in the root of zones isn't "because the standards are old" it's because CNAMEs are supposed to be a one-off which should be used sparingly because they can be used to create never-ending chains of indirection which cause serious problems for nameservers doing recursive resolution for their users. AOL used to regularly break nameservers all over the place with their casual use of endless loops of CNAME records.
- xxdesmus 13y agoAny number of companies these days don't provide you with an A record for the root ...Heroku, Amazon's ELB, etc etc etc ...the list goes on. We're solving a real problem considering how popular those services are these days. Perhaps it's not an issue for your setup, but for anyone using those names servers (among many others) it's a very real issue when you'd typically need to use a CNAME if you want to use the root vs. the antiquated WWW.
- jedberg 13y agoThis is a great example of marketing vs. technology. Cloudflare is amazingly good at marketing. This technology is not new, revolutionary or different in any way from what a bunch of other providers do (including AWS, who they even mention in the blog post as what they are a solution for). But yet they write it like they have solved a major problem in a new way. Super props to the Cloudflare team! (I'm being sincere here, they did a great job with the marketing part of this product)
- earless1 13y agoIt is my understanding that the AWS solution only works when using Route53 with other AWS services like ELB and S3. We had a problem similar in the past and the solution was not pretty. I think they have solved the problem in a fashion that makes it easier for everyone to implement.
- mindslight 13y ago> The problem stems from the fact that the DNS specification dates from 1987 ... As a result, the DNS spec enshrined that the root record -- the naked domain without any subdomain -- could not be a CNAME This isn't due to a lack of foresight in the spec, but people's misunderstanding of what a CNAME record actually means. A CNAME says that this name refers to the exact same thing as another name, including all subnames. You can CNAME the "bare" example.com to foo.example.org, but it would involve getting the .com nameservers serve that record instead of NS records. By having NS and SOA records for example.com, you can no longer say that the two names are equivalent. Rather than having to define a merge algorithm (eg what happens if example.com and foo.example.org both have MX records?), the RFC simply rules out additional records on the alias.
- AnthonyMouse 13y agoWhich makes me wonder what they're doing for record types other than A/AAAA. If you have "example.com. CNAME example.net." then which record does Cloudflare provide if both example.com and example.net each have an MX record, or a TXT record, etc.? Do you get the MX record from example.com if there is one and the MX record from example.net if there isn't?
- takeda 13y agoThey butchered it when allowed CNAME in the root breaking RFC and other things with it, so I would not be surprised that they don't do anything, but perhaps majority only cares about A/AAAA.
- fblp 13y agoThis is a really well written blog post. Great work cloudflare team.
- lingben 13y agoBut why use CNAMEs in the first place when you don't have to? or am I missing something?
- earless1 13y agoWhen using a service like the AWS load balancer all you get back is an A record. You don't have a direct IP address to point to so you have to use a Cname
- mleonhard 13y agoIf you don't want to use CloudFlare as your DNS provider, you can use https://www.rootredirect.com/ https://www.rootredirect.com/ to point your root domain to your ELB or other CNAME-based load balancer. Disclosure: I own rootredirect.com.