6 ms·

UDP and me

166 points by mmastrac 11y ago

tptacek 11y ago
One project where my friend and officemate Steven T. Kent (now chief scientist and vice president at BBN, and a chief advisor to NSA) and I lost was our strong argument to put mandatory end-to-end encryption into TCP It's interesting to read him saying this, because Reed is also one of the co-authors on this foundational paper on Internet design: http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf http://web.mit.edu/Saltzer/www/publications/endtoend/endtoen... ... and a straightforward consequence of the philosophy of the End To End Argument is that encryption doesn't belong in TCP at all --- different applications will have different service models, and E2E tells us not to confine those applications to a lowest common denominator provided by a low level protocol. At any rate, given the evolution of transport cryptography between 1980 and 2010, the lack of mandatory TCP encryption is almost certainly a mercy.
- frikk 11y ago
  No kidding. I can only imagine the headaches if the security drama (like upgrading bit lengths, deprecating ciphers, certificates etc) that keeps SSL on its toes were built into TCP. In this case, simplicity rules IMO.
- dvanduzer 11y ago
  Was there something specific that happened in 2010? I don't know if I've noticed you comment specifically on QUIC, for example, but there do seem to be more recent efforts to embed exactly the right amount of crypto in the transport layer.
- Spooky23 11y ago
  I think end to end encryption would have killed the Internet... Key management would have been a weapon wielded by the telephone company to push whatever proprietary junk they came up with. Plus, we'd be stuck with useless 1980 crypto forever.
  - wmf 11y ago
    I think Jon Postel would have been smarter than to put any authority in the hands of telcos.
drawkbox 11y ago
While I’m honored whenever anybody appreciates important design choices where I’ve been involved, this is not quite right, and it’s a little embarrassing to be the inventor of something so simple. Actually, UDP was “un-designed” by me and others. By this I mean that UDP was the final expression of a process that today we would call “factoring” an overly complex design. Actually he should not be embarrassed. Engineering at the core is taking something complex and making it simple, as simple as can be but not too simple. It takes tons of effort to simplify. That is why most engineers/developers think that complexity shows their knowledge, it is actually the direct reverse of that. But where you try to simplify you will feel currents pulling you the other way and you'll have to swim upstream for a bit. Thank you for inventing UDP and by inventing I mean simplifying something unnecessarily complex into something usable. It has been the key to many game technologies and has made possible and been a base platform for some wonderfully complex things, simply by being simple. Complexity only through simple parts. “Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius — and a lot of courage to move in the opposite direction.” ― Ernst F. Schumacher “Life is really simple, but we insist on making it complicated.” ― Confucius
- vacri 11y ago
  I think a better term than 'simple' is 'appropriate'. Unnecessary guff is inappropriate, as is unnecessary simplicity. By stating that simplicity is the goal, we end up with designers who oversimplify, thinking that that is good design. A tool should be appropriate for use, which naturally includes "but not too simple".
  - sitkack 11y ago
    I think the word you are looking for is 'simplistic'.
  - jacobolus 11y ago
    If you haven’t seen them, I recommend the Clojure guys’ talks about the subject of simplicity. They reached into the etymological history of the word “simple” to pull out its early definition, which is quite precise and IMO tremendously useful in this context, unlike the confused muddle of modern definitions. Rich Hickey, “Simple Made Easy”: http://www.infoq.com/presentations/Simple-Made-Easy http://www.infoq.com/presentations/Simple-Made-Easy Stu Halloway, “Simplicity Ain’t Easy”: https://www.youtube.com/watch?v=cidchWg74Y4 https://www.youtube.com/watch?v=cidchWg74Y4
stephentmcm 11y ago
I dressed up as a UDP packet once… I don’t think anyone got it, but I couldn't tell.
- Istof 11y ago
  you used it in the wrong occasion
  - Istof 11y ago
    Even though this was a joke, UDP does need to be use in the correct applications, I don't know why I am getting down-voted for this...
- ibnudaruaji 11y ago
  Hey, I have a joke about UDP. Though I don't know if anyone got it or not, i couldn't tell.
chetanahuja 11y ago
As a designer of a mobile optimized protocol over UDP, my eternal thanks to David P. Reed for making it possible. What a terrible world it would have been if networking was forever doomed to use TCP (or some other "virtual circuit protocol") as the only way to communicate over the network.
- droopyEyelids 11y ago
  Did you read the article?
  - chetanahuja 11y ago
    Yes. Your point is?
    - chx 11y ago
      That you need spread your "eternal thanks" to a wider group than David P. Reed.
      - chetanahuja 11y ago
        hmm... that's an odd thing to call me out on. But yeah, my eternal thanks to everybody responsible for making sure UDP got baked into the internet infrastructure as a first class citizen right from the beginning.
gwu78 11y ago
"And finally, I and Steve Crocker argued for a more flexible endpoint control over addressing, which got turned into a design (also by me) as "source routing" options, particularly "loose source routing". It should have been the standard address, not an option, because the idea that a federated connection of autonomous networks gets to "decide" routing in a way that can't be overridden by endpoints has made the network less scalable and more fragile than it need be. We lost because of an argument that "addresses were too big" for teletype packets, and the thought that 232 (4 billion) machines would NEVER be part of the Internet. (this also eliminated strong arguments made for longer, hierarchical addresses)." I am glad that my kernel still has a loose source routing compile option and that my #1 UDP client (and #2 TCP client, right after djb's tcpclient) has source routing options. That client is the orginal nc. Those options do not get much use (yet) but they represent history I do not want to forget: what is being discussed in the blog post. Apparently people did believe this was the way routing should be done; I like that thinking. As I see it, assumptions were 1. user is not stupid and 2. user should be allowed at least some control. Given a choice between the (theoretical) "end-to-end" internet and the (actual) third party "policy-based" routed internet, I would take the former. But what do I know? I'm just a dumb user. Short of that, I'll settle for an overlay with source routing. UDP works well enough for making overlays, so thank you Mr. Reed.
- MichaelGG 11y ago
  Does anyone support source routing? Like on the actual Internet? SIP has it,too, one of their many mistakes that has to be dealt with by sysadmins in production.
- StephenFalken 11y ago
  You raise a really interesting point about the relevance of Source Routing. Recently I read a small but significant paper by Carl A. Sunshine [1] (from the beginning of 1977) about Source Routing and some of its implications. This was written at a time when there was still a very active community researching the fundamentals of Computer Networking. He shows how one could build an even simpler and dumber (while still functional) core computer network with no routing infrastructure (pretty amazing concept when you think of it), no global addresses and no global node naming. All that complexity would be transferred to the endpoints, making the whole network even more configurable and adaptable to topology changes. Endpoints would have a more powerful and free participation in that network. The End-to-End Principle taken to its limits. Is there still some deep fundamental research on the feasibility of such a large scale network based on Source Routing ? [1] Source Routing in Computer Networks http://cartap.us/p29-sunshine.pdf http://cartap.us/p29-sunshine.pdf
shurcooL 11y ago
Does anyone know when we're getting access to UDP in browser? I am really excited about the possibility hinted at http://www.w3.org/2012/sysapps/tcp-udp-sockets/ http://www.w3.org/2012/sysapps/tcp-udp-sockets/ and already have plans for how to use it.
- pekk 11y ago
  https://lists.w3.org/Archives/Public/public-sysapps/2015Apr/0001.html https://lists.w3.org/Archives/Public/public-sysapps/2015Apr/... For typical webapps it wouldn't help because of the difficulty traversing firewalls. This seems to be about standardizing facilities for browser-based OSes like FXOS and ChromeOS to make things like IRC clients
- hroi 11y ago
  I am quite worried about this development as someone who has to deal with DDoS traffic on a regular basis. I am starting to feel like UDP and UDP-like protocols should be abolished outright.
dsmithatx 11y ago
For me the value in UDP is that it doesn't answer. Knowing robots aren't constantly making note of VPN I provide is a relief. This in a world where there are many TCP services I do have to worry about. Then again I might be ignorant in my bliss in obscurity. After all network admin is one of a 1000 hats I wear in my role and probably the least paid attention to.
UNIXdev0 11y ago
Nice to someone behave so humbly for a change. He believes, probably more or less accurately, that he along with his collaborators made a minor contribution while perched high atop the shoulders of giants and disclaims accordingly the accolades he considers unwarranted if not embarrassing. I suspect if he were offered a Knighthood or some other ostentatious honor, he'd respectfully turn that down as well--unlike certain other Internet protocol "inventors"...
Animats 11y ago
The original plan for IP was that if you created a new datagram-level protocol, you'd get a new IP protocol number for it. There were a lot of those in the early days.[1] Some were just encapsulation of other protocols - Xerox Parc Universal Protocol (IP protocol #12), Ethernet over IP (#97). Others were for network routing and administration - Exterior Gateway Protocol (#8) and Dynamic Source Routing Protocol (#48). Those were all standardized numbers, and IANA still maintains that list, although most of those protocols are long dead. UDP was a general encapsulation for doing the same thing one level higher. Instead of IP level protocol numbers, the whole thing was repeated one level higher, and there are standard UDP port numbers.[2] At the network level, there's no big gain here, and it adds another header and some overhead. But BSD and the socket interface made a big distinction between software based on existing protocols and ones using the "raw socket" interface. That pushed new work to UDP and away from the IP protocol level. If you want to have fun with your network, crank up something that speaks ISO-TP4 (protocol #29) such as Windows 2000 and see how far the packets get. Or look at your corporate network's packet traffic and see if there's any activity on the old protocol numbers. [1] http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml http://www.iana.org/assignments/protocol-numbers/protocol-nu... [2] http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml http://www.iana.org/assignments/service-names-port-numbers/s...
allcentury 11y ago
I started reading his phd thesis he linked in that post, my favorite quote so far: "The term distributed computing has been used to describe the loosely coupled systems built using this technology. But like many other fashionable terms, distributed computing means different things to different users of the term"
revelation 11y ago
If you look at the UDP protocol, it has all of one field in it, the port number. It is only designed in the way that it explicitly has no design.
- js2 11y ago
  Source port, destination port, length, and checksum.
  - FullyFunctional 11y ago
    Nope, you are including the IP header. The UDP head has: port.
    - js2 11y ago
      No, I'm not. Where are you coming up with this? Just go read http://tools.ietf.org/html/rfc768 http://tools.ietf.org/html/rfc768 There's four fields in the UDP header.
jwr 11y ago
I found this bit interesting, given how long ago these events unfolded: > [...] was our strong argument to put mandatory end-to-end encryption into TCP (and adaptations of the ideas to UDP-based protocols, such as RTP, hich I worked out but abandoned). Steve’s design was rejected, not because it was unsound, but because NSA did not want to see ANY encryption work going on in the public domain ARPA project, some say because they did not want to see the world be “too secure” by default.

166 points by mmastrac 11y ago

One project where my friend and officemate Steven T. Kent (now chief scientist and vice president at BBN, and a chief advisor to NSA) and I lost was our strong argument to put mandatory end-to-end encryption into TCP It's interesting to read him saying this, because Reed is also one of the co-authors on this foundational paper on Internet design: http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf http://web.mit.edu/Saltzer/www/publications/endtoend/endtoen... ... and a straightforward consequence of the philosophy of the End To End Argument is that encryption doesn't belong in TCP at all --- different applications will have different service models, and E2E tells us not to confine those applications to a lowest common denominator provided by a low level protocol. At any rate, given the evolution of transport cryptography between 1980 and 2010, the lack of mandatory TCP encryption is almost certainly a mercy.

frikk 11y ago
No kidding. I can only imagine the headaches if the security drama (like upgrading bit lengths, deprecating ciphers, certificates etc) that keeps SSL on its toes were built into TCP. In this case, simplicity rules IMO.
dvanduzer 11y ago
Was there something specific that happened in 2010? I don't know if I've noticed you comment specifically on QUIC, for example, but there do seem to be more recent efforts to embed exactly the right amount of crypto in the transport layer.
Spooky23 11y ago
I think end to end encryption would have killed the Internet... Key management would have been a weapon wielded by the telephone company to push whatever proprietary junk they came up with. Plus, we'd be stuck with useless 1980 crypto forever.
- wmf 11y ago
  I think Jon Postel would have been smarter than to put any authority in the hands of telcos.

drawkbox 11y ago

While I’m honored whenever anybody appreciates important design choices where I’ve been involved, this is not quite right, and it’s a little embarrassing to be the inventor of something so simple. Actually, UDP was “un-designed” by me and others. By this I mean that UDP was the final expression of a process that today we would call “factoring” an overly complex design. Actually he should not be embarrassed. Engineering at the core is taking something complex and making it simple, as simple as can be but not too simple. It takes tons of effort to simplify. That is why most engineers/developers think that complexity shows their knowledge, it is actually the direct reverse of that. But where you try to simplify you will feel currents pulling you the other way and you'll have to swim upstream for a bit. Thank you for inventing UDP and by inventing I mean simplifying something unnecessarily complex into something usable. It has been the key to many game technologies and has made possible and been a base platform for some wonderfully complex things, simply by being simple. Complexity only through simple parts. “Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius — and a lot of courage to move in the opposite direction.” ― Ernst F. Schumacher “Life is really simple, but we insist on making it complicated.” ― Confucius

vacri 11y ago
I think a better term than 'simple' is 'appropriate'. Unnecessary guff is inappropriate, as is unnecessary simplicity. By stating that simplicity is the goal, we end up with designers who oversimplify, thinking that that is good design. A tool should be appropriate for use, which naturally includes "but not too simple".
- sitkack 11y ago
  I think the word you are looking for is 'simplistic'.
- jacobolus 11y ago
  If you haven’t seen them, I recommend the Clojure guys’ talks about the subject of simplicity. They reached into the etymological history of the word “simple” to pull out its early definition, which is quite precise and IMO tremendously useful in this context, unlike the confused muddle of modern definitions. Rich Hickey, “Simple Made Easy”: http://www.infoq.com/presentations/Simple-Made-Easy http://www.infoq.com/presentations/Simple-Made-Easy Stu Halloway, “Simplicity Ain’t Easy”: https://www.youtube.com/watch?v=cidchWg74Y4 https://www.youtube.com/watch?v=cidchWg74Y4

stephentmcm 11y ago

I dressed up as a UDP packet once… I don’t think anyone got it, but I couldn't tell.

Istof 11y ago
you used it in the wrong occasion
- Istof 11y ago
  Even though this was a joke, UDP does need to be use in the correct applications, I don't know why I am getting down-voted for this...
ibnudaruaji 11y ago
Hey, I have a joke about UDP. Though I don't know if anyone got it or not, i couldn't tell.

chetanahuja 11y ago

As a designer of a mobile optimized protocol over UDP, my eternal thanks to David P. Reed for making it possible. What a terrible world it would have been if networking was forever doomed to use TCP (or some other "virtual circuit protocol") as the only way to communicate over the network.

droopyEyelids 11y ago
Did you read the article?
- chetanahuja 11y ago
  Yes. Your point is?
  - chx 11y ago
    That you need spread your "eternal thanks" to a wider group than David P. Reed.
    - chetanahuja 11y ago
      hmm... that's an odd thing to call me out on. But yeah, my eternal thanks to everybody responsible for making sure UDP got baked into the internet infrastructure as a first class citizen right from the beginning.

gwu78 11y ago

"And finally, I and Steve Crocker argued for a more flexible endpoint control over addressing, which got turned into a design (also by me) as "source routing" options, particularly "loose source routing". It should have been the standard address, not an option, because the idea that a federated connection of autonomous networks gets to "decide" routing in a way that can't be overridden by endpoints has made the network less scalable and more fragile than it need be. We lost because of an argument that "addresses were too big" for teletype packets, and the thought that 232 (4 billion) machines would NEVER be part of the Internet. (this also eliminated strong arguments made for longer, hierarchical addresses)." I am glad that my kernel still has a loose source routing compile option and that my #1 UDP client (and #2 TCP client, right after djb's tcpclient) has source routing options. That client is the orginal nc. Those options do not get much use (yet) but they represent history I do not want to forget: what is being discussed in the blog post. Apparently people did believe this was the way routing should be done; I like that thinking. As I see it, assumptions were 1. user is not stupid and 2. user should be allowed at least some control. Given a choice between the (theoretical) "end-to-end" internet and the (actual) third party "policy-based" routed internet, I would take the former. But what do I know? I'm just a dumb user. Short of that, I'll settle for an overlay with source routing. UDP works well enough for making overlays, so thank you Mr. Reed.

MichaelGG 11y ago
Does anyone support source routing? Like on the actual Internet? SIP has it,too, one of their many mistakes that has to be dealt with by sysadmins in production.
StephenFalken 11y ago
You raise a really interesting point about the relevance of Source Routing. Recently I read a small but significant paper by Carl A. Sunshine [1] (from the beginning of 1977) about Source Routing and some of its implications. This was written at a time when there was still a very active community researching the fundamentals of Computer Networking. He shows how one could build an even simpler and dumber (while still functional) core computer network with no routing infrastructure (pretty amazing concept when you think of it), no global addresses and no global node naming. All that complexity would be transferred to the endpoints, making the whole network even more configurable and adaptable to topology changes. Endpoints would have a more powerful and free participation in that network. The End-to-End Principle taken to its limits. Is there still some deep fundamental research on the feasibility of such a large scale network based on Source Routing ? [1] Source Routing in Computer Networks http://cartap.us/p29-sunshine.pdf http://cartap.us/p29-sunshine.pdf

shurcooL 11y ago

Does anyone know when we're getting access to UDP in browser? I am really excited about the possibility hinted at http://www.w3.org/2012/sysapps/tcp-udp-sockets/ http://www.w3.org/2012/sysapps/tcp-udp-sockets/ and already have plans for how to use it.

pekk 11y ago
https://lists.w3.org/Archives/Public/public-sysapps/2015Apr/0001.html https://lists.w3.org/Archives/Public/public-sysapps/2015Apr/... For typical webapps it wouldn't help because of the difficulty traversing firewalls. This seems to be about standardizing facilities for browser-based OSes like FXOS and ChromeOS to make things like IRC clients
hroi 11y ago
I am quite worried about this development as someone who has to deal with DDoS traffic on a regular basis. I am starting to feel like UDP and UDP-like protocols should be abolished outright.

dsmithatx 11y ago

For me the value in UDP is that it doesn't answer. Knowing robots aren't constantly making note of VPN I provide is a relief. This in a world where there are many TCP services I do have to worry about. Then again I might be ignorant in my bliss in obscurity. After all network admin is one of a 1000 hats I wear in my role and probably the least paid attention to.

UNIXdev0 11y ago

Nice to someone behave so humbly for a change. He believes, probably more or less accurately, that he along with his collaborators made a minor contribution while perched high atop the shoulders of giants and disclaims accordingly the accolades he considers unwarranted if not embarrassing. I suspect if he were offered a Knighthood or some other ostentatious honor, he'd respectfully turn that down as well--unlike certain other Internet protocol "inventors"...

Animats 11y ago

The original plan for IP was that if you created a new datagram-level protocol, you'd get a new IP protocol number for it. There were a lot of those in the early days.[1] Some were just encapsulation of other protocols - Xerox Parc Universal Protocol (IP protocol #12), Ethernet over IP (#97). Others were for network routing and administration - Exterior Gateway Protocol (#8) and Dynamic Source Routing Protocol (#48). Those were all standardized numbers, and IANA still maintains that list, although most of those protocols are long dead. UDP was a general encapsulation for doing the same thing one level higher. Instead of IP level protocol numbers, the whole thing was repeated one level higher, and there are standard UDP port numbers.[2] At the network level, there's no big gain here, and it adds another header and some overhead. But BSD and the socket interface made a big distinction between software based on existing protocols and ones using the "raw socket" interface. That pushed new work to UDP and away from the IP protocol level. If you want to have fun with your network, crank up something that speaks ISO-TP4 (protocol #29) such as Windows 2000 and see how far the packets get. Or look at your corporate network's packet traffic and see if there's any activity on the old protocol numbers. [1] http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml http://www.iana.org/assignments/protocol-numbers/protocol-nu... [2] http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml http://www.iana.org/assignments/service-names-port-numbers/s...

allcentury 11y ago

I started reading his phd thesis he linked in that post, my favorite quote so far: "The term distributed computing has been used to describe the loosely coupled systems built using this technology. But like many other fashionable terms, distributed computing means different things to different users of the term"

revelation 11y ago

If you look at the UDP protocol, it has all of one field in it, the port number. It is only designed in the way that it explicitly has no design.

js2 11y ago
Source port, destination port, length, and checksum.
- FullyFunctional 11y ago
  Nope, you are including the IP header. The UDP head has: port.
  - js2 11y ago
    No, I'm not. Where are you coming up with this? Just go read http://tools.ietf.org/html/rfc768 http://tools.ietf.org/html/rfc768 There's four fields in the UDP header.

jwr 11y ago

I found this bit interesting, given how long ago these events unfolded: > [...] was our strong argument to put mandatory end-to-end encryption into TCP (and adaptations of the ideas to UDP-based protocols, such as RTP, hich I worked out but abandoned). Steve’s design was rejected, not because it was unsound, but because NSA did not want to see ANY encryption work going on in the public domain ARPA project, some say because they did not want to see the world be “too secure” by default.