5 ms·

Having a variable header complicates Cut-Through switching[0], which forwards the entire frame just after the switch gets the destination field of the header.

Having a variable header complicates Cut-Through switching[0], which forwards the entire frame just after the switch gets the destination field of the header. This sounds like it would force a return to Store-And-Forward[1], where the switch waits for the entire frame to load into memory, before forwarding to its next hop. Waiting these few milliseconds doesn't sound bad, until you consider that's added to each packet, reducing (e.g.) application performance. [0] https://en.wikipedia.org/wiki/Cut-through_switching https://en.wikipedia.org/wiki/Cut-through_switching [1] https://en.wikipedia.org/wiki/Store_and_forward https://en.wikipedia.org/wiki/Store_and_forward

squeed 9y ago
The real problem is that you can no longer look at a fixed offset for your parameters, which complicates filtering silicon. In other words, if you want to have a firewall rule of "drop tcp 135", in ipv4 you can just look at bytes 0x24-25 (for Ethernet). IPv6's extension-header mechanism means that the header has a non-fixed length, so you need to do work for the same effect. Given that firewalls are critical in a world without NAT, this can be a scary prospect. Part of the blame falls on vendors who go "well, IPv6 is fringe so we'll just do all firewalling in software." The protocol is 20 years old, it's time to design the silicon.
- tssva 9y ago
  Most enterprise firewalls these days do some level of layer 7 packet inspection which requires going beyond looking at fixed offsets. The silicon to support this already exists. Even the $50 Ubiquiti ER-X router I use at home can do deep packet inspection assisted by silicon.
- verri 9y ago
  But IPv4 headers could have a variable length too, it's just that we don't encounter that much in the wild. And what about the AH header, GRE and IPIP tunnel headers, 6rd/6in4 tunnel headers, etc. Filtering at fixed offsets sounds very brittle to me. Isn't the real problem here that network providers and administrators appropriate the right to filter on OSI layers they shouldn't be touching? This problem sounds like the exact reason why Google insisted on having its QUIC headers ciphered: so network equipment can't pull of this kind of misbehaviour.
  - noselasd 9y ago
    At least with IPv4 you can easily compute the start of layer 4, it's just ipv4[0]&0xf * 4 - trivially implemented with little real-estate in hardware. IPv6 requires you to loop through all extension headers to reach layer 4, and you need to know about a handful of those extension headers as not all follow the same format.
tssva 9y ago
Totally different layers. Switching occurs at layer 2 and isn't affected by whether the frame contains IPv4 or IPv6.
- wmf 9y ago
  It would be nice if that was the case, but switches often look deeper for ECMP.
- stephengillie 9y ago
  I was confused at first, because switches were mentioned in the article. But the Fates have allowed my mistake to be correct this one time, and provided a trove of education about the state of the art of network switching.
teddyh 9y ago
You seem to be confusing IP headers with Ethernet headers. Cut-through switching only looks at the Ethernet header, which is not variable, and thus will still be perfectly viable.
wmf 9y ago
The way cut-through is really implemented is that the first N bytes of the packet (usually 128 or 256) are treated as the "header" and are passed through the header pipeline regardless of how big the actual header is. So variable headers like VLAN tags, MPLS stacks, or IP extension headers shouldn't prevent cut-through. Also, the difference between cut-through and store-and-forward is generally in single-digit microseconds, not milliseconds. A good rule of thumb is ~1 us/KB at 10G.
- myrandomcomment 9y ago
  So there was a huge argument of CT vs SF switching way back when. IIRC there was Kalpana (CT) which Cisco acquired and Granite (Cat4K, SF) and Crescendo (Cat5/6K, SF). Cisco killed cut-through. The 1st real ASIC that did cut-through since then was the FM4000 (Fulcum) which was used in the Arista 7124, BNT, etc. The FM4000 latency was 600ns vs. 6 microseconds for the standard Wall Street switch at the time (Cat4900). 600ns to 6 microseconds is huge. The FM4000 looked at 64 bytes before forwarding (IIRC). The 1st Broadcom chips that did cut-through were the Trident line. They looked at something like ~200 or ~500 bytes before making a forwarding decision (sorry exact number is not in cache anymore). They cut that down with the T+ chips. The funny issue about cut-through is that it will forward a corrupted packet. Trying to pinpoint the source can be fun. With static webpages, etc, old school internet the latency was not a big deal. The improvements in switching with cut-through and low latency were a game changer for storage, video, etc.

squeed 9y ago

The real problem is that you can no longer look at a fixed offset for your parameters, which complicates filtering silicon. In other words, if you want to have a firewall rule of "drop tcp 135", in ipv4 you can just look at bytes 0x24-25 (for Ethernet). IPv6's extension-header mechanism means that the header has a non-fixed length, so you need to do work for the same effect. Given that firewalls are critical in a world without NAT, this can be a scary prospect. Part of the blame falls on vendors who go "well, IPv6 is fringe so we'll just do all firewalling in software." The protocol is 20 years old, it's time to design the silicon.

tssva 9y ago
Most enterprise firewalls these days do some level of layer 7 packet inspection which requires going beyond looking at fixed offsets. The silicon to support this already exists. Even the $50 Ubiquiti ER-X router I use at home can do deep packet inspection assisted by silicon.
verri 9y ago
But IPv4 headers could have a variable length too, it's just that we don't encounter that much in the wild. And what about the AH header, GRE and IPIP tunnel headers, 6rd/6in4 tunnel headers, etc. Filtering at fixed offsets sounds very brittle to me. Isn't the real problem here that network providers and administrators appropriate the right to filter on OSI layers they shouldn't be touching? This problem sounds like the exact reason why Google insisted on having its QUIC headers ciphered: so network equipment can't pull of this kind of misbehaviour.
- noselasd 9y ago
  At least with IPv4 you can easily compute the start of layer 4, it's just ipv4[0]&0xf * 4 - trivially implemented with little real-estate in hardware. IPv6 requires you to loop through all extension headers to reach layer 4, and you need to know about a handful of those extension headers as not all follow the same format.

tssva 9y ago

Totally different layers. Switching occurs at layer 2 and isn't affected by whether the frame contains IPv4 or IPv6.

wmf 9y ago
It would be nice if that was the case, but switches often look deeper for ECMP.
stephengillie 9y ago
I was confused at first, because switches were mentioned in the article. But the Fates have allowed my mistake to be correct this one time, and provided a trove of education about the state of the art of network switching.

teddyh 9y ago

You seem to be confusing IP headers with Ethernet headers. Cut-through switching only looks at the Ethernet header, which is not variable, and thus will still be perfectly viable.

wmf 9y ago

The way cut-through is really implemented is that the first N bytes of the packet (usually 128 or 256) are treated as the "header" and are passed through the header pipeline regardless of how big the actual header is. So variable headers like VLAN tags, MPLS stacks, or IP extension headers shouldn't prevent cut-through. Also, the difference between cut-through and store-and-forward is generally in single-digit microseconds, not milliseconds. A good rule of thumb is ~1 us/KB at 10G.

myrandomcomment 9y ago
So there was a huge argument of CT vs SF switching way back when. IIRC there was Kalpana (CT) which Cisco acquired and Granite (Cat4K, SF) and Crescendo (Cat5/6K, SF). Cisco killed cut-through. The 1st real ASIC that did cut-through since then was the FM4000 (Fulcum) which was used in the Arista 7124, BNT, etc. The FM4000 latency was 600ns vs. 6 microseconds for the standard Wall Street switch at the time (Cat4900). 600ns to 6 microseconds is huge. The FM4000 looked at 64 bytes before forwarding (IIRC). The 1st Broadcom chips that did cut-through were the Trident line. They looked at something like ~200 or ~500 bytes before making a forwarding decision (sorry exact number is not in cache anymore). They cut that down with the T+ chips. The funny issue about cut-through is that it will forward a corrupted packet. Trying to pinpoint the source can be fun. With static webpages, etc, old school internet the latency was not a big deal. The improvements in switching with cut-through and low latency were a game changer for storage, video, etc.