3 ms·

RDMA may bypass the CPU but it doesn't bypass PCIe. It still goes through it, which is slower than NVLink on Power9 (~16GB/s on 16x PCIe 3.0 vs ~150GB/s on NVLi

by arnon 8y ago

RDMA may bypass the CPU but it doesn't bypass PCIe. It still goes through it, which is slower than NVLink on Power9 (~16GB/s on 16x PCIe 3.0 vs ~150GB/s on NVLink). Data has to be copied up to the GPU for processing. It has to be brought down to be sent as a result set somewhere. If everything you're doing fits in GPU memory, you've only done that once. If it doesn't, you need to copy data up and down. You may need to do that for intermediate results too. This adds up.

shaklee3 8y ago
It doesn't matter if it bypasses pcie. The fastest available ethernet interface today is 100Gbps. Pcie gen3 x16 will get you to about 115Gbps, so the limiter is the interface speed. Pcie4 should be common when 200G interfaces come out. This is why an external nvswitch can make network switches vulnerable in this market. Power 9 has pcie4, so you could theoretically already double your pcie bandwidth with a mellanox card that supports it, but from my previous point, it doesn't matter. Those nvlink numbers are disingenuous. That's the total links to/from a GPU. All available power 9 machines split those links between the other GPUs and CPUs, so you really only have up to 3 of those in the best case, or 75GBps. That's only a bit more than double pcie 4 at that point. And also, by using those nvlinks back to the processor, you are making a decision that you don't need as much bandwidth between GPUs. With the nvswitch architecture, you get the best of both worlds with a full, non-blocking switch.
twtw 8y ago
This is true, but it is essentially the same situation as using the CPU for processing.
janfrode 8y ago
There’s actually six 50 GB/s nvlinks on each Power9 cpu (300GB/s), 2 or 3 to each GPU, depending on if you have a 4 or 6 GPU system. I’ve seen speculation that the DGX-2 nvswitches has enough free ports to fully connect in 2 Power9 cpus with 300 GB/s each, that would be an interesting system..

by arnon 8y ago

shaklee3 8y ago

It doesn't matter if it bypasses pcie. The fastest available ethernet interface today is 100Gbps. Pcie gen3 x16 will get you to about 115Gbps, so the limiter is the interface speed. Pcie4 should be common when 200G interfaces come out. This is why an external nvswitch can make network switches vulnerable in this market. Power 9 has pcie4, so you could theoretically already double your pcie bandwidth with a mellanox card that supports it, but from my previous point, it doesn't matter. Those nvlink numbers are disingenuous. That's the total links to/from a GPU. All available power 9 machines split those links between the other GPUs and CPUs, so you really only have up to 3 of those in the best case, or 75GBps. That's only a bit more than double pcie 4 at that point. And also, by using those nvlinks back to the processor, you are making a decision that you don't need as much bandwidth between GPUs. With the nvswitch architecture, you get the best of both worlds with a full, non-blocking switch.

twtw 8y ago

This is true, but it is essentially the same situation as using the CPU for processing.

janfrode 8y ago

There’s actually six 50 GB/s nvlinks on each Power9 cpu (300GB/s), 2 or 3 to each GPU, depending on if you have a 4 or 6 GPU system. I’ve seen speculation that the DGX-2 nvswitches has enough free ports to fully connect in 2 Power9 cpus with 300 GB/s each, that would be an interesting system..