I did not benchmark. This is theoretical and talks about specific conditions: when your training is bottlenecked on communication between GPUs. PCIE 4.0 is exac

I did not benchmark. This is theoretical and talks about specific conditions: when your training is bottlenecked on communication between GPUs. PCIE 4.0 is exactly 2x bandwidth, and PCIE 5.0 is another 2x more.