4 ms·
RDMA is common for high performance applications but it doesn't work over the Internet.
by fragmede 2y ago
RDMA is common for high performance applications but it doesn't work over the Internet.
- Danieru 2y agoIt's a good thing the NIC is connected over pcie then.
- shaklee3 2y agoYou can do GPUdirect over the Internet without RDMA though.
- jpgvm 2y agoGPUDirect relies on the PeerDirect extensions for RDMA and are thus an extension to the RDMA verbs, not a separate an independent thing that works without RDMA.
- shaklee3 2y agoAgain, you can do what I said. You may be using different terminology, but you can do GPUdirect in dpdk without rdma
- jpgvm 2y agoYou can read/write to GPU buffers with gpudev in DPDK yes. It also uses some of the infrastructure that powers GPUDirect (namely the page pinning and address translation). Because you can use the addressable memory in DPDK buffer steering you can have the NIC DMA to/from the GPU and then have a GPU kernel coordinate with your DPDK application. This will be pretty fast on a good lossless datacentre network but probably pretty awful over the Internet. In the DC though it will be beaten by real GPUDirect on RDMA naturally as you don't need the DPDK coordinator and all tx/rx can be driven by the GPU kernel instead. This isn't GPUDirect though, that is an actual product.
- shaklee3 2y agoThis is GPUDirect. GPUDirect is the technology that enables any third-party device to talk to a GPU (like a NIC). > but probably pretty awful over the Internet. In the DC though it will be beaten by real GPUDirect on RDMA naturally It's being used in many places successfully over the internet. RDMA is fine, but completely breaks the abstraction of services. In many places you do not want to know who is sending or what address to send/receive.
- throw0101c 2y ago> RDMA is common for high performance applications but it doesn't work over the Internet. RoCEv2 is routable. * https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet * https://docs.nvidia.com/networking/display/winofv55053000/rocev2 https://docs.nvidia.com/networking/display/winofv55053000/ro... Of course you're going to get horrible latency because of speed-of-light limitations, so the definition of "work" may be weak, but data should be able to be transmitted.