4 ms·

Interesting to see this paper making HN. I think it has stood the test of time reasonably well - it was written in about four hours one morning under rather un

Interesting to see this paper making HN. I think it has stood the test of time reasonably well - it was written in about four hours one morning under rather unfortunate circumstances - I was stuck on a train that hit a vehicle at a crossing. I'm interested in what HN folks think has changed. What are today's urgent problems, and what are just annoying ones that will cost time and money to work around, but aren't critical? How do they differ from those of 10 years ago? Arguably, some I discussed back then have only grown in importance, with no solution in sight - for example, DDoS attacks. Whereas some, like address space exhaustion, look like being adequately addressed, as IPv6 is finally rolling out (only took 20 years!).

quanticle 10y ago
I'm wondering if your opinions on the difficulty of introducing new protocols have changed over the past decade. For example, Bittorrent has taken off, and while I used to have to manually configure ports on my NAT to allow Bittorrent to pass through, I no longer need to do so. Universal PnP seems to have taken care of that (at the cost of some security).
- mhandley 10y ago
  STUN/TURN/ICE have enabled NAT hole-punching to become fairly reliable, but the complexity involved is horrible. Try reading RFC 5245 and its companions - this isn't a sane way to design an Internet. It does work though. New protocols are still an issue, but the problems differ depending on whether we're taking about the transport layer or the application layer. We've tried and largely failed to deploy alternatives to TCP and UDP. My effort, DCCP, for multimedia traffic is dead in the water due in part to NAT/Firewall traversal problems, and is now essentially tunneled over UDP. SCTP is successful in some limited circles, but you're very unlikely to see it end-to-end across the Internet for the same reasons. Even TCP is really hard to extend. If you're interested, we did some work on this a few years ago: http://www0.cs.ucl.ac.uk/staff/M.Handley/papers/extend-tcp.pdf http://www0.cs.ucl.ac.uk/staff/M.Handley/papers/extend-tcp.p... It's amazing what some middleboxes do to TCP. At the application layer, it's somewhat easier. You can deploy new protocols over UDP, as BitTorrent did and as Google has done with QUIC, and they'll work in most places, but you'll still need a fallback. The universal fallback these days seems to be to tunnel over TLS on TCP port 443, because that's rarely blocked. The fact that we need to do this is an admission of failure on the part of the Internet architecture. The fact that it just about works is the reason we've not needed to address the underlying problem, which is the tension between security and extensibility.
gone35 10y ago
Security, if you ask me.
- mhandley 10y ago
  No disagreement there - see our work on standardizing tcpcrypt in the IETF tcpinc working group for our attempts to improve one small corner of this space. https://datatracker.ietf.org/wg/tcpinc/charter/ https://datatracker.ietf.org/wg/tcpinc/charter/
telesilla 10y ago
Did anyone think about applications with time-sensitive arrival back then (e.g. VoIP), and have any implementation ideas that never made it out the door?
- mhandley 10y ago
  I've been working on VoIP and Internet multimedia since 1991 (I'm one of the authors of SIP, for my sins) so, yes, we've been concerned about time-sensitive material for a long time. From a standards point of view, this resulted in first, Interv and RSVP, then diffserv. The former largely failed, mostly for economic reasons, while the latter is widely used, but almost never for end-to-end service, again for economic reasons. Basically, if you're going to give priority to some traffic, there needs to be a disincentive for all traffic to demand priority, which usually means charging. No-one ever figured out a good economic model for end-to-end QoS, and most users aren't willing to pay extra anyway. There has been a lot of effort more recently on sensible queue management schemes to avoid buffer-bloat, which show significant promise. For low-speed customer links, variants of fair queuing generally work well. My ISP, Andrews and Arnold, is one of the few that really gets this. They have a shaper, upstream of the DSL or FTTC link, that can be configured to 95% of the negotiated link speed, so there are no queues downstream. Then they run fairly small buffers in the shaper and WFQ. I pretty much never see any significant latency on my VoIP calls due to queuing. In general, low-latency VoIP shouldn't be a problem - so long as the data rate is lower then the rate of the mean flow on a link, you can give it strict priority service without harming the other traffic. In practice, this requires some sort of policing to avoid abuse. Various forms of fair queuing approximate this - a flow that has a low data rate will be serviced frequently (though not as quickly as it it had priority), and see pretty low latency in the queue on a busy link. Now, getting low latency for high quality video conferencing; that's a different question. Fair queuing doesn't really help, because the data rate can be relatively high. Instead, you've got to keep the total queue size small. This can impact competing TCP flows if you're not careful, and can cause non-negligible packet loss. Thus is due to TCP's congestion control dynamics - throughput of a TCP flow is inversely proportional to round trip time, and inversely proportional to square root of loss rate. If you reduce queuing at the bottleneck, you reduce RTT, so push up the loss rate. This is one of the reasons people gripe about TCP's congestion control. However, if ECN were widely deployed, this would become a non-issue, because we wouldn't need those packet drops to control TCP's sending rate. Generally, widely deploying ECN would be the single biggest enabler for low-latency networking.
jandrese 10y ago
The fragility of BGP is one area that I have been concerned about for some time. We have already seen instances of misconfiguration that black hole entire countries, but if someone wanted to bring down the Internet intentionally they could do a lot of damage with just a few hacked ISPs.
ComodoHacker 10y ago
Net neutrality is one of the problems that wasn't envisioned (at least in your paper) 10 years ago. DDoS problem becomes pressing now with the emergence of IoT. Still no general solution though.

quanticle 10y ago

I'm wondering if your opinions on the difficulty of introducing new protocols have changed over the past decade. For example, Bittorrent has taken off, and while I used to have to manually configure ports on my NAT to allow Bittorrent to pass through, I no longer need to do so. Universal PnP seems to have taken care of that (at the cost of some security).

mhandley 10y ago
STUN/TURN/ICE have enabled NAT hole-punching to become fairly reliable, but the complexity involved is horrible. Try reading RFC 5245 and its companions - this isn't a sane way to design an Internet. It does work though. New protocols are still an issue, but the problems differ depending on whether we're taking about the transport layer or the application layer. We've tried and largely failed to deploy alternatives to TCP and UDP. My effort, DCCP, for multimedia traffic is dead in the water due in part to NAT/Firewall traversal problems, and is now essentially tunneled over UDP. SCTP is successful in some limited circles, but you're very unlikely to see it end-to-end across the Internet for the same reasons. Even TCP is really hard to extend. If you're interested, we did some work on this a few years ago: http://www0.cs.ucl.ac.uk/staff/M.Handley/papers/extend-tcp.pdf http://www0.cs.ucl.ac.uk/staff/M.Handley/papers/extend-tcp.p... It's amazing what some middleboxes do to TCP. At the application layer, it's somewhat easier. You can deploy new protocols over UDP, as BitTorrent did and as Google has done with QUIC, and they'll work in most places, but you'll still need a fallback. The universal fallback these days seems to be to tunnel over TLS on TCP port 443, because that's rarely blocked. The fact that we need to do this is an admission of failure on the part of the Internet architecture. The fact that it just about works is the reason we've not needed to address the underlying problem, which is the tension between security and extensibility.

gone35 10y ago

Security, if you ask me.

mhandley 10y ago
No disagreement there - see our work on standardizing tcpcrypt in the IETF tcpinc working group for our attempts to improve one small corner of this space. https://datatracker.ietf.org/wg/tcpinc/charter/ https://datatracker.ietf.org/wg/tcpinc/charter/

telesilla 10y ago

Did anyone think about applications with time-sensitive arrival back then (e.g. VoIP), and have any implementation ideas that never made it out the door?

mhandley 10y ago
I've been working on VoIP and Internet multimedia since 1991 (I'm one of the authors of SIP, for my sins) so, yes, we've been concerned about time-sensitive material for a long time. From a standards point of view, this resulted in first, Interv and RSVP, then diffserv. The former largely failed, mostly for economic reasons, while the latter is widely used, but almost never for end-to-end service, again for economic reasons. Basically, if you're going to give priority to some traffic, there needs to be a disincentive for all traffic to demand priority, which usually means charging. No-one ever figured out a good economic model for end-to-end QoS, and most users aren't willing to pay extra anyway. There has been a lot of effort more recently on sensible queue management schemes to avoid buffer-bloat, which show significant promise. For low-speed customer links, variants of fair queuing generally work well. My ISP, Andrews and Arnold, is one of the few that really gets this. They have a shaper, upstream of the DSL or FTTC link, that can be configured to 95% of the negotiated link speed, so there are no queues downstream. Then they run fairly small buffers in the shaper and WFQ. I pretty much never see any significant latency on my VoIP calls due to queuing. In general, low-latency VoIP shouldn't be a problem - so long as the data rate is lower then the rate of the mean flow on a link, you can give it strict priority service without harming the other traffic. In practice, this requires some sort of policing to avoid abuse. Various forms of fair queuing approximate this - a flow that has a low data rate will be serviced frequently (though not as quickly as it it had priority), and see pretty low latency in the queue on a busy link. Now, getting low latency for high quality video conferencing; that's a different question. Fair queuing doesn't really help, because the data rate can be relatively high. Instead, you've got to keep the total queue size small. This can impact competing TCP flows if you're not careful, and can cause non-negligible packet loss. Thus is due to TCP's congestion control dynamics - throughput of a TCP flow is inversely proportional to round trip time, and inversely proportional to square root of loss rate. If you reduce queuing at the bottleneck, you reduce RTT, so push up the loss rate. This is one of the reasons people gripe about TCP's congestion control. However, if ECN were widely deployed, this would become a non-issue, because we wouldn't need those packet drops to control TCP's sending rate. Generally, widely deploying ECN would be the single biggest enabler for low-latency networking.

jandrese 10y ago

The fragility of BGP is one area that I have been concerned about for some time. We have already seen instances of misconfiguration that black hole entire countries, but if someone wanted to bring down the Internet intentionally they could do a lot of damage with just a few hacked ISPs.

ComodoHacker 10y ago

Net neutrality is one of the problems that wasn't envisioned (at least in your paper) 10 years ago. DDoS problem becomes pressing now with the emergence of IoT. Still no general solution though.