4 ms·
get a GH100 on lambda and behold you have 900GB/s between CPU memory and GPU, and forget PCIe.
by _zoltan_ 2y ago
get a GH100 on lambda and behold you have 900GB/s between CPU memory and GPU, and forget PCIe.
- saagarjha 2y agoWhere are you seeing 900 GB/s?
- KeplerBoy 2y agoThe 900 GB/s are a figure often cited for Hopper based SXM boards, it's the aggregate of 18 NVLink connections. So it's more of a many-to-many GPU-to-GPU bandwidth figure. https://www.datacenterknowledge.com/data-center-hardware/nvidia-s-h100-what-it-is-what-it-does-and-why-it-matters https://www.datacenterknowledge.com/data-center-hardware/nvi...
- shaklee3 2y agothis is bidirectional bw and it's gh200, not gh100
- imtringued 2y agoThat doesn't add up, because you're now exceeding the memory bandwidth of the memory controller. I.e. it would be faster to do everything, including the CPU only algorithms, in far away VRAM.
- gpderetta 2y agolatency might be higher and latency usually dominates CPU side algorithms.