And PCIe Gen 3 is limited to 16GB/s, which is the real bottleneck between the CPU and GPU... if you can use all 16 lanes. Most GPGPU set ups with multiple GPUs

And PCIe Gen 3 is limited to 16GB/s, which is the real bottleneck between the CPU and GPU... if you can use all 16 lanes. Most GPGPU set ups with multiple GPUs only are able to use 4 or 8 lanes each, so you then are stuck with multiple 4 or 8GB/s bottlenecks.