6 ms·

I don't find the wording in the RFC to be that ambiguous actually. > The answer to the query, possibly preface by one or more CNAME RRs that specify aliases en

by steve1977 9mo ago

I don't find the wording in the RFC to be that ambiguous actually. > The answer to the query, possibly preface by one or more CNAME RRs that specify aliases encountered on the way to an answer. The "possibly preface" (sic!) to me is obviously to be understood as "if there are any CNAME RRs, the answer to the query is to be prefaced by those CNAME RRs" and not "you can preface the query with the CNAME RRs or you can place them wherever you want".

paulddraper 9mo ago
100% I just commented the same. It's pretty clear that the "possibly" refers to the presence of the CNAME RRs, not the ordering.
- Dylan16807 9mo ago
  The context makes it less clear, but even if we pretend that part is crystal, a comment that stops there is missing the point of the article. All CNAMEs at the start isn't enough. The order of the CNAMEs can cause problems despite perfect RFC compliance.
- andrewshadura 9mo ago
  To me, this reads exactly the opposite.
a7b3fa 9mo ago
I agree with you, and I also think that their interpretation of example 6.2.1 in the RFC is somewhat nonsensical. It states that “The difference in ordering of the RRs in the answer section is not significant.” But from the RFC, very clearly this comment is relevant only to that particular example; it is comparing two responses and saying that in this case, the different ordering has no semantic effect. And perhaps this is somewhat pedantic, but they also write that “RFC 1034 section 3.6 defines Resource Record Sets (RRsets) as collections of records with the same name, type, and class.” But looking at the RFC, it never defines such a term; it does say that within a “set” of RRs “associated with a particular name” the order doesn’t matter. But even if the RFC had said “associated with a particular combination of name, type, and class”, I don’t see how that could have introduced ambiguity. It specifies an exception to a general rule, so obviously if the exception doesn’t apply, then the general rule must be followed. Anyway, Cloudflare probably know their DNS better than I do, but I did not find the article especially persuasive; I think the ambiguity is actually just a misreading, and that the RFC does require a particular ordering of CNAME records. (ETA:) Although admittedly, while the RFC does say that CNAMEs must come before As in the answer, I don’t necessarily see any clear rule about how CNAME chains must be ordered; the RFC just says “Domain names in RRs which point at another name should always point at the primary name and not the alias ... Of course, by the robustness principle, domain software should not fail when presented with CNAME chains or loops; CNAME chains should be followed”. So actually I guess I do agree that there is some ambiguity about the responses containing CNAME chains.
inopinatus 9mo ago
The article makes it very clear that the ambiguity arises in another phrase: “difference in ordering of the RRs in the answer section is not significant”, which is applied to an example; the problem with examples being that they are illustrative, viz. generalisable, and thus may permit reordering everywhere, and in any case, whether they should or shouldn’t becomes a matter of pragmatic context. Which goes to show, one person’s “obvious understanding” is another’s “did they even read the entire document”. All of which also serves to highlight the value of normative language, but that came later.
- PunchyHamster 9mo ago
  it wouldn't be a problem if they tested it properly... especially WHEN stuff is ambigous
  - nraynaud 9mo ago
    They may not have realized their interpretation is ambiguous until after the incident, that’s the kind of stuff you realize after you find a bug and do a deep dive in the literature for a post mortem. They probably worked with the certitude that record order is irrelevant until that point.
taeric 9mo ago
Isn't this literally noted in the article? The article even points out that the RFC is from before normative words were standardized for hard requirements.
mrmattyboy 9mo ago
I agree this doens't seem too ambiguous - it's "you may do this.." and they said "or we may do the reverse". If I say you're could prefix something.. the alternative isn't that you can suffix it. But also.. the programmers working on the software running one of the most important (end-user) DNS servers in the world: 1. Changes logic in how CNAME responses are formed 2. I assume some tests at least broke that meant they needed to be "fixed up" (y'know - "when a CNAME is queried, I expect this response") 3. No one saw these changes in test behavoir and thought "I wonder if this order is important". Or "We should research more into this", Or "Are other DNS servers changing order", Or "This should be flagged for a very gradual release". 4. Ends up in test environment for, what, a month.. nothing using getaddrinfo from glibc is being used to test this environment or anyone noticed that it was broken Cloudflare seem to be getting into thr swing of breaking things and then being transparent. But this really reads as a fun "did you know", not a "we broke things again - please still use us". There's no real RCA except to blame an RFC - but honestly, for a large-scale operation like there's this seems very big to slip through the cracks. I would make a joke about South Park's oil "I'm sorry".. but they don't even seem to be
- bpt3 9mo ago
  > Ends up in test environment for, what, a month.. nothing using getaddrinfo from glibc is being used to test this environment or anyone noticed that it was broken This is the part that is shocking to me. How is getaddrinfo not called in any unit or system tests?
  - SAI_Peregrinus 9mo ago
    Probably Alpine containers, so musl's version instead of glibc's.
  - zinekeller 9mo ago
    As black3r mentioned (https://news.ycombinator.com/item?id=46686096 https://news.ycombinator.com/item?id=46686096), it is likely rearranged by systemd, therefore only non-systemd glibc distributions are affected. I would hazard a guess that their test environment have both the systemd variant and the Unbound variants (Unbound technically does not arrange them, but instead reconstructs it according to RFC "CNAME restart" logic because it is a recursive resolver in itself), but not just plain directly-piped resolv.conf (Presumably because who would run that in this day and age. This is sadly just a half-joke, because only a few people would fall on this category.)
devman0 9mo ago
Even if 'possibly preface' is interpreted to mean CNAME RRSets should appear first there is still a broken reliance by some resolvers on the order of CNAME RRsets if there is more than one CNAME in the chain. This expectation of ordering is not promised by the relevant RFCs.
the_mitsuhiko 9mo ago
> I don't find the wording in the RFC to be that ambiguous actually. You might not find it ambiguous but it is ambiguous and there were attempts to fix it. You can find a warmed up discussion about this topic here: https://mailarchive.ietf.org/arch/msg/dnsop/2USkYvbnSIQ8s2vfTL4PrCXeUmM/ https://mailarchive.ietf.org/arch/msg/dnsop/2USkYvbnSIQ8s2vf...
VoodooJuJu 9mo ago
[dead]
bigstrat2003 9mo ago
My initial reading was "you can place them wherever you want". And given that multiple parties are naturally interpreting the wording in different ways, that means the wording is ambiguous by definition.

paulddraper 9mo ago

100% I just commented the same. It's pretty clear that the "possibly" refers to the presence of the CNAME RRs, not the ordering.

Dylan16807 9mo ago
The context makes it less clear, but even if we pretend that part is crystal, a comment that stops there is missing the point of the article. All CNAMEs at the start isn't enough. The order of the CNAMEs can cause problems despite perfect RFC compliance.
andrewshadura 9mo ago
To me, this reads exactly the opposite.

a7b3fa 9mo ago

I agree with you, and I also think that their interpretation of example 6.2.1 in the RFC is somewhat nonsensical. It states that “The difference in ordering of the RRs in the answer section is not significant.” But from the RFC, very clearly this comment is relevant only to that particular example; it is comparing two responses and saying that in this case, the different ordering has no semantic effect. And perhaps this is somewhat pedantic, but they also write that “RFC 1034 section 3.6 defines Resource Record Sets (RRsets) as collections of records with the same name, type, and class.” But looking at the RFC, it never defines such a term; it does say that within a “set” of RRs “associated with a particular name” the order doesn’t matter. But even if the RFC had said “associated with a particular combination of name, type, and class”, I don’t see how that could have introduced ambiguity. It specifies an exception to a general rule, so obviously if the exception doesn’t apply, then the general rule must be followed. Anyway, Cloudflare probably know their DNS better than I do, but I did not find the article especially persuasive; I think the ambiguity is actually just a misreading, and that the RFC does require a particular ordering of CNAME records. (ETA:) Although admittedly, while the RFC does say that CNAMEs must come before As in the answer, I don’t necessarily see any clear rule about how CNAME chains must be ordered; the RFC just says “Domain names in RRs which point at another name should always point at the primary name and not the alias ... Of course, by the robustness principle, domain software should not fail when presented with CNAME chains or loops; CNAME chains should be followed”. So actually I guess I do agree that there is some ambiguity about the responses containing CNAME chains.

inopinatus 9mo ago

The article makes it very clear that the ambiguity arises in another phrase: “difference in ordering of the RRs in the answer section is not significant”, which is applied to an example; the problem with examples being that they are illustrative, viz. generalisable, and thus may permit reordering everywhere, and in any case, whether they should or shouldn’t becomes a matter of pragmatic context. Which goes to show, one person’s “obvious understanding” is another’s “did they even read the entire document”. All of which also serves to highlight the value of normative language, but that came later.

PunchyHamster 9mo ago
it wouldn't be a problem if they tested it properly... especially WHEN stuff is ambigous
- nraynaud 9mo ago
  They may not have realized their interpretation is ambiguous until after the incident, that’s the kind of stuff you realize after you find a bug and do a deep dive in the literature for a post mortem. They probably worked with the certitude that record order is irrelevant until that point.

taeric 9mo ago

Isn't this literally noted in the article? The article even points out that the RFC is from before normative words were standardized for hard requirements.

mrmattyboy 9mo ago

I agree this doens't seem too ambiguous - it's "you may do this.." and they said "or we may do the reverse". If I say you're could prefix something.. the alternative isn't that you can suffix it. But also.. the programmers working on the software running one of the most important (end-user) DNS servers in the world: 1. Changes logic in how CNAME responses are formed 2. I assume some tests at least broke that meant they needed to be "fixed up" (y'know - "when a CNAME is queried, I expect this response") 3. No one saw these changes in test behavoir and thought "I wonder if this order is important". Or "We should research more into this", Or "Are other DNS servers changing order", Or "This should be flagged for a very gradual release". 4. Ends up in test environment for, what, a month.. nothing using getaddrinfo from glibc is being used to test this environment or anyone noticed that it was broken Cloudflare seem to be getting into thr swing of breaking things and then being transparent. But this really reads as a fun "did you know", not a "we broke things again - please still use us". There's no real RCA except to blame an RFC - but honestly, for a large-scale operation like there's this seems very big to slip through the cracks. I would make a joke about South Park's oil "I'm sorry".. but they don't even seem to be

bpt3 9mo ago
> Ends up in test environment for, what, a month.. nothing using getaddrinfo from glibc is being used to test this environment or anyone noticed that it was broken This is the part that is shocking to me. How is getaddrinfo not called in any unit or system tests?
- SAI_Peregrinus 9mo ago
  Probably Alpine containers, so musl's version instead of glibc's.
- zinekeller 9mo ago
  As black3r mentioned (https://news.ycombinator.com/item?id=46686096 https://news.ycombinator.com/item?id=46686096), it is likely rearranged by systemd, therefore only non-systemd glibc distributions are affected. I would hazard a guess that their test environment have both the systemd variant and the Unbound variants (Unbound technically does not arrange them, but instead reconstructs it according to RFC "CNAME restart" logic because it is a recursive resolver in itself), but not just plain directly-piped resolv.conf (Presumably because who would run that in this day and age. This is sadly just a half-joke, because only a few people would fall on this category.)

devman0 9mo ago

Even if 'possibly preface' is interpreted to mean CNAME RRSets should appear first there is still a broken reliance by some resolvers on the order of CNAME RRsets if there is more than one CNAME in the chain. This expectation of ordering is not promised by the relevant RFCs.

the_mitsuhiko 9mo ago

> I don't find the wording in the RFC to be that ambiguous actually. You might not find it ambiguous but it is ambiguous and there were attempts to fix it. You can find a warmed up discussion about this topic here: https://mailarchive.ietf.org/arch/msg/dnsop/2USkYvbnSIQ8s2vfTL4PrCXeUmM/ https://mailarchive.ietf.org/arch/msg/dnsop/2USkYvbnSIQ8s2vf...

VoodooJuJu 9mo ago

[dead]

bigstrat2003 9mo ago

My initial reading was "you can place them wherever you want". And given that multiple parties are naturally interpreting the wording in different ways, that means the wording is ambiguous by definition.