3 ms·
> GPU interconnects for direct GPU to GPU communication Note that nvlink is also available on the 3090, which does also have the same bus width as the A6000.
by my123 5y ago
> GPU interconnects for direct GPU to GPU communication
Note that nvlink is also available on the 3090, which does also have the same bus width as the A6000.
- justinlloyd 5y agoSo interestingly to get multiple RTX 3090 cards using NVLINK you really need the blower style, which NVidia recently stopped making in the consumer line, otherwise you are fighting thermal properties and getting in to exotic cooling solutions. And you also get in to placement in slots on the motherboard on three-slot cards. And then you get into the number of PCIe PSU connectors on most supplies. You also get in to weird corner cases of four way NVLINK on 3090's acting as a "single card" but four way NVLINK on RTX A5000 is still independent cards as far as the OS goes. Regarding NVLINK, 3090 cards only support NVLINK (NVL 3.0 IIRC), rather than NVSWITCH. With NVLINK/NVSWITCH, with two RTX A5000 cards I get a total of 48GB of usable RAM for storing the machine learning models, and so on for multiple cards in the RTX A5K & A6K & AX00 line. But with the consumer cards, even though each card has, e.g. 12GB of RAM on them, your deep learning model is also limited to 12GB of RAM, no matter how many 3090 cards you put in. You're also limited to a maximum of four 3090 cards in total, whereas with the NVLINK & NVSWITCH on the professional grade cards, you are limited only by how many cards you can stuff in your data center - technically it is 16 GPUs on a single switch IIRC, things get funky with the fabric once you progress beyond 16 GPUs on a single switch but that's outside of the scope of this conversation. I may also have some details wrong, my brain is a little fuzzy afer dinner.