4 ms·

It’s been shown that leaving holes in the sequence space isn’t good for middleboxes [0]. Wanting TCP to behave more like UDP is a reasonable goal (e.g., for rea

It’s been shown that leaving holes in the sequence space isn’t good for middleboxes [0]. Wanting TCP to behave more like UDP is a reasonable goal (e.g., for reachability), but you throw the benefits away if 20% of connections are disrupted by middleboxes. Other approaches, like Minion [1] or TCP Hollywood [2], see deployability because they adhere to the assumptions made about TCP by middleboxes. [0]: “Is it Still Possible to Extend TCP?” - http://www0.cs.ucl.ac.uk/staff/ucacmha/papers/extend-tcp.pdf http://www0.cs.ucl.ac.uk/staff/ucacmha/papers/extend-tcp.pdf [1]: Minion - https://tools.ietf.org/html/draft-iyengar-minion-protocol-01 https://tools.ietf.org/html/draft-iyengar-minion-protocol-01 [2]: TCP Hollywood - http://ieeexplore.ieee.org/document/7497221/ http://ieeexplore.ieee.org/document/7497221/

fulafel 9y ago
It would be very good for the net if something came up that penalised middleboxes though. Maybe this could be it.
- lumisota 9y ago
  QUIC demonstrates a better approach: encrypt everything, limiting the ability for middleboxes to make assumptions about behaviour.
  - kevincox 9y ago
    Even QUIC found that the couple of bytes of "flag" fields in the header were enough for middleboxes to mess with it. It seems incredibly hard to devise a protocol which is extensible without having middleboxes mess it up.
    - lumisota 9y ago
      Indeed. Though that issue sort of validated the approach: middlebox vendors will latch on to any unencrypted bits they can. The only way to stop that is to encrypt _everything_.
- zrm 9y ago
  > It would be very good for the net if something came up that penalised middleboxes though. What you can do is to have a good protocol that requires no interference from middleboxes but detects it if it happens, and then a less efficient legacy fallback protocol that basically looks as much as possible like HTTPS. Then if you detect interference from a middlebox, show the user a message that says, "WARNING: MAN IN THE MIDDLE ATTACK DETECTED. Something is modifying connections on this network. This may compromise security and performance." Then hopefully having multiple different apps show a message like that to every user on the network will get enough users complaining to fix the middlebox so that it stops breaking new things.
zlynx 9y ago
Yes indeed. If I could crash middle-boxes by sending them malformed TCP streams, I would do it all day long.
bostonvaulter2 9y ago
What are middleboxes? This is something that I've never heard of before.
- doctorsher 9y ago
  Middleboxes are machines somewhere along the path (when you're sending to a particular destination) that drop, filter, bandwidth shape, or otherwise edit your traffic. Basically anything besides packet forwarding.
- mhandley 9y ago
  Anything on the path between the endpoints that looks at layers above IP. Obvious examples are NATs, firewalls, transparent proxies, traffic normalizers, so-called protocol accelerators. Some boxes I've no idea what they do. Take a look at our paper in reference [0] above, which probes what they actually do to traffic.
- staunch 9y ago
  Middlebox is a technically imprecise term that is not in common usage among technical people anywhere. Why people in this thread are trotting it out is anyone's guess.
  - detaro 9y ago
    That's just plain wrong, in the context of protocol development it is a commonly used term, and not exactly uncommon in wider networking space as well. See MPTCP mentions in this thread, presentations and mailing lists around development of TLS, HTTP2, QUIC, ...
  - deleted 9y ago
    [deleted]
  - dfox 9y ago
    Middlebox has quite precise technical meaning: anything that works at higher layer than on which it is visible to neighboring network. Usualy this refers to various "security" "solutions" which attempt to do deep packet inspection and generally break various things, but it also can mean NAT or L2 bridges that do filtering on L3/L4 headers (for example DOCSIS CMs and CMTSes are such middleboxes)
mhandley 9y ago
Yes, we were surprised by just how prevalent middleboxes were that didn't like gaps in sequence spaces. This significantly influenced the design options for MPTCP. Another problem is that TCP is a bytestream protocol. Apps that stream over TCP don't usually add packet-orientated framing and resync points, so if you lose a packet, the receiver will often need to discard quite a bit of data after the missing packet before they can start decoding again. Effectively this multiplies the effective loss rate. In the extreme, there's the potential for congestion collapse, where lots of packets are being delivered but none of the are useful, so they're all discarded at the receiver. Edit: I should add - middleboxes often resegment the datastream, merging multiple packets, or splitting large ones. So even if the sender added a header in each segment sent, those headers may not be at the beginning of the segment when it arrives. After a loss, you may not be able to reliable find the next header again. By the way, that web server at UCL may well be the oldest on the Internet. It's probably the only server left proudly running CERN/3.0 on Sparc hardware since 1994.
- lumisota 9y ago
  Your point about resegmentation (in combination with loss or out-of-order delivery) is why a framing layer, like the one in Minion, is necessary. The Minion paper [3] does a good job of illustrating the problem. [3]: https://arxiv.org/abs/1103.0463 https://arxiv.org/abs/1103.0463

fulafel 9y ago

It would be very good for the net if something came up that penalised middleboxes though. Maybe this could be it.

lumisota 9y ago
QUIC demonstrates a better approach: encrypt everything, limiting the ability for middleboxes to make assumptions about behaviour.
- kevincox 9y ago
  Even QUIC found that the couple of bytes of "flag" fields in the header were enough for middleboxes to mess with it. It seems incredibly hard to devise a protocol which is extensible without having middleboxes mess it up.
  - lumisota 9y ago
    Indeed. Though that issue sort of validated the approach: middlebox vendors will latch on to any unencrypted bits they can. The only way to stop that is to encrypt _everything_.
zrm 9y ago
> It would be very good for the net if something came up that penalised middleboxes though. What you can do is to have a good protocol that requires no interference from middleboxes but detects it if it happens, and then a less efficient legacy fallback protocol that basically looks as much as possible like HTTPS. Then if you detect interference from a middlebox, show the user a message that says, "WARNING: MAN IN THE MIDDLE ATTACK DETECTED. Something is modifying connections on this network. This may compromise security and performance." Then hopefully having multiple different apps show a message like that to every user on the network will get enough users complaining to fix the middlebox so that it stops breaking new things.

zlynx 9y ago

Yes indeed. If I could crash middle-boxes by sending them malformed TCP streams, I would do it all day long.

bostonvaulter2 9y ago

What are middleboxes? This is something that I've never heard of before.

doctorsher 9y ago
Middleboxes are machines somewhere along the path (when you're sending to a particular destination) that drop, filter, bandwidth shape, or otherwise edit your traffic. Basically anything besides packet forwarding.
mhandley 9y ago
Anything on the path between the endpoints that looks at layers above IP. Obvious examples are NATs, firewalls, transparent proxies, traffic normalizers, so-called protocol accelerators. Some boxes I've no idea what they do. Take a look at our paper in reference [0] above, which probes what they actually do to traffic.
staunch 9y ago
Middlebox is a technically imprecise term that is not in common usage among technical people anywhere. Why people in this thread are trotting it out is anyone's guess.
- detaro 9y ago
  That's just plain wrong, in the context of protocol development it is a commonly used term, and not exactly uncommon in wider networking space as well. See MPTCP mentions in this thread, presentations and mailing lists around development of TLS, HTTP2, QUIC, ...
- deleted 9y ago
  [deleted]
- dfox 9y ago
  Middlebox has quite precise technical meaning: anything that works at higher layer than on which it is visible to neighboring network. Usualy this refers to various "security" "solutions" which attempt to do deep packet inspection and generally break various things, but it also can mean NAT or L2 bridges that do filtering on L3/L4 headers (for example DOCSIS CMs and CMTSes are such middleboxes)

mhandley 9y ago

Yes, we were surprised by just how prevalent middleboxes were that didn't like gaps in sequence spaces. This significantly influenced the design options for MPTCP. Another problem is that TCP is a bytestream protocol. Apps that stream over TCP don't usually add packet-orientated framing and resync points, so if you lose a packet, the receiver will often need to discard quite a bit of data after the missing packet before they can start decoding again. Effectively this multiplies the effective loss rate. In the extreme, there's the potential for congestion collapse, where lots of packets are being delivered but none of the are useful, so they're all discarded at the receiver. Edit: I should add - middleboxes often resegment the datastream, merging multiple packets, or splitting large ones. So even if the sender added a header in each segment sent, those headers may not be at the beginning of the segment when it arrives. After a loss, you may not be able to reliable find the next header again. By the way, that web server at UCL may well be the oldest on the Internet. It's probably the only server left proudly running CERN/3.0 on Sparc hardware since 1994.

lumisota 9y ago
Your point about resegmentation (in combination with loss or out-of-order delivery) is why a framing layer, like the one in Minion, is necessary. The Minion paper [3] does a good job of illustrating the problem. [3]: https://arxiv.org/abs/1103.0463 https://arxiv.org/abs/1103.0463