4 ms·

If you're in a data center and running large training jobs then RDMA over Nvidia Mellanox Infiniband cards over high speed ethernet (like 100GB) are used to shi

If you're in a data center and running large training jobs then RDMA over Nvidia Mellanox Infiniband cards over high speed ethernet (like 100GB) are used to ship coefficients around without having that transfer bottleneck in the CPU.

xxpor 3y ago
100 gig, that's considered cute nowadays. https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/ https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances... 3.2 terabits.
- dekhn 3y ago
  I don't think that machine has a single nic with that bandwidth- I'd guess it's 8 400Gbps cards or something similar.
  - asciimike 3y ago
    Correct, it's 8x 400G cards, one per GPU.