4 ms·

And why is "big endian vs little endian" a thing in computers and networking? Because it's homage to Jonathan Swift. It references an apocryphal war about which

by m3047 3y ago

And why is "big endian vs little endian" a thing in computers and networking? Because it's homage to Jonathan Swift. It references an apocryphal war about which end of a hard-boiled egg to lop off; no doubt a much more vicious affair than the Haiti / Dominican spat over the pronunciation of "parsely". Parts of the internet are still stuck in the 1980s, my pet peeve being the DNS resolution protocol which calls on recursing resolvers to try TCP after receiving a successful UDP response with TC=1. (Where did this become "only"? When did it become ok not to support TCP?) Seems like IP works just fine on top of ATM. Maybe the MTU isn't the same as ethernet; too bad. By the way UDP frags are bad; don't do that. Guess it's not completely transparent afterall and relying on a kludge mechanism as the answer to knowing the terrain (MTU) isn't the One True Way. TCP is a reliability mechanism built on top of IP. It has enough popularity to have its own protocol designator so that you can't say it's "built on top of UDP". Don't want that mechanism? Don't use it. Want messaging? Use a datagram protocol; that would be UDP (unless you decide to use your own special protocol ID). It's unreliable? Do something about it. (See where I'm going with this?) Craft your messages so that they fit in a single packet. Invest in some decent networking gear that reports dropped packets and allows traffic shaping. Bonus: get one-to-many and transit-based destination editing. If you refuse to compromise on your abastraction and are too cheap to afford SDN, create a virtualization layer for your network traffic like everybody before you... it's virtual turtles all the way down or hasn't that occurred to you?

phicoh 3y ago
You may want to talk to operators of DNS servers to find out why DNS is mostly UDP. From a software point of view is hardly matters. UDP has some quirks, TCP has some issues, not a big deal. The big deal for operators is resource consumption and latency. In the vast majority of cases where the UDP reply is not truncated, using UDP is both cheaper and faster.
- m3047 3y ago
  I have worked for Paul Vixie. (I've met Mockepetris and Liu.) Even Vixie agrees UDP frags were a bad idea for The DNS. Given the context of where we're discussing this, mind if I ask: Why is UDP so attractive for DNS operators? Is it that it encapsulates the message query / response paradigm so well? Why isn't encryption of greater concern? How can DoT or DoH be "faster" than UDP? Is it really the protocol or a misattribution of causality actually based on LFNs, buffer bloat, and mis- or mal- construction of the tenet that UDP traffic can be dropped with no consequence (instead of employing traffic shaping, for instance). Honestly, lay it on me: explain to me about messaging versus pipes using DNS as an example.
  - phicoh 3y ago
    Fragmentation is a complex subject. If avoiding fragmentation was obviously good, then UDP packets would always be sent with the don't fragment (DF) bit set. In reality, that would make some situation much worse than allowing fragmentation to happen. DoT and DoH are certainly not faster than UDP. It is only in the long tail that DoT and DoH work better. If encryption is required for DNS, then obviously UDP and plain TCP not suitable. But operators of authoritative DNS servers are very reluctant to support DoT. The IETF is still trying to hash out how to use DoT between recursive resolvers and authoritative server.
    - m3047 3y ago
      Thanks. Apropos "virtual turtles all the way down" I was expecting something along the lines of the well-recognized historical practice of site to site VPNS and tunnelling for DNS backhaul. It's not fair to conflate "DNS operators" with the relationship between authoritative and recursive operators. The (main) problem with encryption has to do with context. In order to achieve privacy, key and/or secret material needs to be negotiated between the ends and this context needs to be retrievable for reference. There is no provision in the DNS for that. There is no provision in the transport per se either. I argue that this is appropriate and correct, that generating and negotiating this material is a separate concern and developing the protocols to do so proceeds on a different track (QED). A better term than "connection" for the concept we are about to discuss would be "circuit". Once in possession of the approprate key material (and the out of band signaling this implies) it's possible for the parties to exchange individually encrypted messages in the open, but this ignores network issues such as dropped packets and fragmentation to name a couple. This could be dealt with but it is not a general solution. If you're going to solve problems it's generally accepted that it's prudent to solve them generally: hence the notion of a circuit. (Frags and dropped packets are other general problems to be solved by a circuit implementation.) It's easy to see that such a generalized circuit is a tunnel and that UDP (or TCP) packets of arbitrary size (up to architectural limits, but significantly larger than either ATM or ethernet frames) can transit such a tunnel with guarantees of no loss and no fragmentation. DNS specifies a very lightweight convention of its own for encapsulating UDP traffic in a TCP tunnel: the payloads are prepended with with a (2 byte) length word [0]. The tunnel protocol (carefully) doesn't even specify that responses via this tunnel need to preserve the ordering of requests. Bearing in mind that context needs to be preserved for each (active) request, the context required for the tunnel itself is amortized by the number of requests using the particular tunnel instance or circuit. (No need to worry about frags, all messages are preserved up to the architectural limit specified by the underlying DNS protocol.) So, this is one example of solving the message-vs-stream dilemma in the real world. We can see that IP-over-IP is a real thing and that DNS provides a point solution for eliding frame size issues but that more general solutions also exist which also support e.g. encryption. The key architectural consideration is mostly who maintains the circuit context and where it is maintained. [0] Yes of course the redundant IP header is omitted for the encapsulated messages.

by m3047 3y ago

phicoh 3y ago

You may want to talk to operators of DNS servers to find out why DNS is mostly UDP. From a software point of view is hardly matters. UDP has some quirks, TCP has some issues, not a big deal. The big deal for operators is resource consumption and latency. In the vast majority of cases where the UDP reply is not truncated, using UDP is both cheaper and faster.

m3047 3y ago
I have worked for Paul Vixie. (I've met Mockepetris and Liu.) Even Vixie agrees UDP frags were a bad idea for The DNS. Given the context of where we're discussing this, mind if I ask: Why is UDP so attractive for DNS operators? Is it that it encapsulates the message query / response paradigm so well? Why isn't encryption of greater concern? How can DoT or DoH be "faster" than UDP? Is it really the protocol or a misattribution of causality actually based on LFNs, buffer bloat, and mis- or mal- construction of the tenet that UDP traffic can be dropped with no consequence (instead of employing traffic shaping, for instance). Honestly, lay it on me: explain to me about messaging versus pipes using DNS as an example.
- phicoh 3y ago
  Fragmentation is a complex subject. If avoiding fragmentation was obviously good, then UDP packets would always be sent with the don't fragment (DF) bit set. In reality, that would make some situation much worse than allowing fragmentation to happen. DoT and DoH are certainly not faster than UDP. It is only in the long tail that DoT and DoH work better. If encryption is required for DNS, then obviously UDP and plain TCP not suitable. But operators of authoritative DNS servers are very reluctant to support DoT. The IETF is still trying to hash out how to use DoT between recursive resolvers and authoritative server.
  - m3047 3y ago
    Thanks. Apropos "virtual turtles all the way down" I was expecting something along the lines of the well-recognized historical practice of site to site VPNS and tunnelling for DNS backhaul. It's not fair to conflate "DNS operators" with the relationship between authoritative and recursive operators. The (main) problem with encryption has to do with context. In order to achieve privacy, key and/or secret material needs to be negotiated between the ends and this context needs to be retrievable for reference. There is no provision in the DNS for that. There is no provision in the transport per se either. I argue that this is appropriate and correct, that generating and negotiating this material is a separate concern and developing the protocols to do so proceeds on a different track (QED). A better term than "connection" for the concept we are about to discuss would be "circuit". Once in possession of the approprate key material (and the out of band signaling this implies) it's possible for the parties to exchange individually encrypted messages in the open, but this ignores network issues such as dropped packets and fragmentation to name a couple. This could be dealt with but it is not a general solution. If you're going to solve problems it's generally accepted that it's prudent to solve them generally: hence the notion of a circuit. (Frags and dropped packets are other general problems to be solved by a circuit implementation.) It's easy to see that such a generalized circuit is a tunnel and that UDP (or TCP) packets of arbitrary size (up to architectural limits, but significantly larger than either ATM or ethernet frames) can transit such a tunnel with guarantees of no loss and no fragmentation. DNS specifies a very lightweight convention of its own for encapsulating UDP traffic in a TCP tunnel: the payloads are prepended with with a (2 byte) length word [0]. The tunnel protocol (carefully) doesn't even specify that responses via this tunnel need to preserve the ordering of requests. Bearing in mind that context needs to be preserved for each (active) request, the context required for the tunnel itself is amortized by the number of requests using the particular tunnel instance or circuit. (No need to worry about frags, all messages are preserved up to the architectural limit specified by the underlying DNS protocol.) So, this is one example of solving the message-vs-stream dilemma in the real world. We can see that IP-over-IP is a real thing and that DNS provides a point solution for eliding frame size issues but that more general solutions also exist which also support e.g. encryption. The key architectural consideration is mostly who maintains the circuit context and where it is maintained. [0] Yes of course the redundant IP header is omitted for the encapsulated messages.