Indeed, it seems that 80usec is just given as an upper bound based on the 1MB buffer at 100Gbps. It is definitely possible to go much lower than 80usec on Ethe

Indeed, it seems that 80usec is just given as an upper bound based on the 1MB buffer at 100Gbps. It is definitely possible to go much lower than 80usec on Ethernet. But obviously it depends on the scale, utilisation etc. At the sizes of GPU clusters we're talking about these days - 32K and up - things get tricky. The main alternative to Infiniband used in the industry is RoCE - Meta has written a lot about it [0]. There's several reasons to avoid Infiniband, such as cost, availability, vendor lock in, lack of experience etc. Those are some of the reasons why many players are trying hard to make Ethernet work, see Ultra Ethernet [1]. [0] https://engineering.fb.com/2024/08/05/data-center-engineering/roce-network-distributed-ai-training-at-scale/ https://engineering.fb.com/2024/08/05/data-center-engineerin... [1] https://ultraethernet.org/ https://ultraethernet.org/