4 ms·
You can, for instance, connect two RTX 3090 with an NVLink bridge. That gives you 48 GB in total. The 4090 doesn't support NVLink anymore.
by rerx 4y ago
You can, for instance, connect two RTX 3090 with an NVLink bridge. That gives you 48 GB in total. The 4090 doesn't support NVLink anymore.
- koheripbal 4y ago> The 4090 doesn't support NVLink anymore. Are you sure about that?
- homarp 4y agothat's what the press said: https://www.tomshardware.com/news/gigabyte-leaves-nvlink-traces-on-geforce-rtx-4090 https://www.tomshardware.com/news/gigabyte-leaves-nvlink-tra...
- mk_stjames 4y agoYou actually can split a model [0] onto multiple GPUs even without NVLink, just using the PCIe for the transfers. Depending on the model the performance is sometimes not all that different. I believe for solely inference on some models the speed difference may barely be noticeable, where for other training activities it may make 10+% difference [1] [0] https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html#single-machine-model-parallel-best-practices https://pytorch.org/tutorials/intermediate/model_parallel_tu... [1] https://huggingface.co/transformers/v4.9.2/performance.html https://huggingface.co/transformers/v4.9.2/performance.html