4 ms·

Consider that a QSFP28 module uses four 25gbps lanes to support sending one single 100gbps flow. So electronics do exist that can easily do what you are asking

Consider that a QSFP28 module uses four 25gbps lanes to support sending one single 100gbps flow. So electronics do exist that can easily do what you are asking. I think it is just the economics of doing it for the various ports on a switch, lack of a standard, etc.

Polizeiposaune 1y ago
SFP/QSFP/PCIe etc., are combining multiple lanes originating from a physical bundle of limited size; transmitters could easily share a single clock source. The wire protocol includes additional signalling that lets the receiver know how to recombine the bits coming over each lane in the correct order. In contrast, Ethernet link aggregation lets you combine ports that can be arbitrarily far apart -- maybe not even within the same rack (see MC-LAG). Ethernet link aggregation doesn't add any encapsulation or sequencing information to the data flows it manages. You can imagine an alternate mechanism which added a small header to each packet with sequence numbers; the other end of the aggregation would then have to remove that header after sorting the packets in order..
BobbyTables2 1y ago
Plus the NIC/PHY is likely assuming only a small range of propagation delay differences between the lanes/links. Probably falls down if one link is 1cm and the other is 100km. A LAG could be done with different medium/speeds, though perhaps not likely in practice.
- eqvinox 1y ago
  > A LAG could be done with different medium/speeds, though perhaps not likely in practice. 802.1AX indeed requires all members of a LAG to use the same speed.
- ryao 1y ago
  You can calculate that using the deskew buffer sizes from here: https://www.ieee802.org/3/df/public/23_01/0130/ran_3df_03_230130.pdf https://www.ieee802.org/3/df/public/23_01/0130/ran_3df_03_23... That said, I fully expect such an order of magnitude difference to overwhelm the deskew buffer.

Polizeiposaune 1y ago

SFP/QSFP/PCIe etc., are combining multiple lanes originating from a physical bundle of limited size; transmitters could easily share a single clock source. The wire protocol includes additional signalling that lets the receiver know how to recombine the bits coming over each lane in the correct order. In contrast, Ethernet link aggregation lets you combine ports that can be arbitrarily far apart -- maybe not even within the same rack (see MC-LAG). Ethernet link aggregation doesn't add any encapsulation or sequencing information to the data flows it manages. You can imagine an alternate mechanism which added a small header to each packet with sequence numbers; the other end of the aggregation would then have to remove that header after sorting the packets in order..

BobbyTables2 1y ago

Plus the NIC/PHY is likely assuming only a small range of propagation delay differences between the lanes/links. Probably falls down if one link is 1cm and the other is 100km. A LAG could be done with different medium/speeds, though perhaps not likely in practice.

eqvinox 1y ago
> A LAG could be done with different medium/speeds, though perhaps not likely in practice. 802.1AX indeed requires all members of a LAG to use the same speed.
ryao 1y ago
You can calculate that using the deskew buffer sizes from here: https://www.ieee802.org/3/df/public/23_01/0130/ran_3df_03_230130.pdf https://www.ieee802.org/3/df/public/23_01/0130/ran_3df_03_23... That said, I fully expect such an order of magnitude difference to overwhelm the deskew buffer.