3 ms·
Probably much faster. PCIe is the limiting factor in a 3x A100 setup. Having all that memory directly accessible by the GPU will make a massive difference. Even
by josephg 3y ago
Probably much faster. PCIe is the limiting factor in a 3x A100 setup. Having all that memory directly accessible by the GPU will make a massive difference. Even the massive 480 gigs of cpu memory in this architecture can be accessed many times faster than pcie allows.
- touisteur 3y agoHopefully if you have 3x A100 you have bridged them with NVLink and that's your bottleneck. If you want more bandwith you'll need a NVSwitch. Or a H100 GPU (Lovelace didn't get NVLink, not even the A40-little-NVLink-port - or PCIe5 for that matter) with next gen NVLink.