4 ms·
I'm super duper curious if there are ways to glob together VRAM between consumer-grade hardware to make this whole market more accessible to the common hacker?
by aliljet 4y ago
I'm super duper curious if there are ways to glob together VRAM between consumer-grade hardware to make this whole market more accessible to the common hacker?
- rerx 4y agoYou can, for instance, connect two RTX 3090 with an NVLink bridge. That gives you 48 GB in total. The 4090 doesn't support NVLink anymore.
- koheripbal 4y ago> The 4090 doesn't support NVLink anymore. Are you sure about that?
- homarp 4y agothat's what the press said: https://www.tomshardware.com/news/gigabyte-leaves-nvlink-traces-on-geforce-rtx-4090 https://www.tomshardware.com/news/gigabyte-leaves-nvlink-tra...
- mk_stjames 4y agoYou actually can split a model [0] onto multiple GPUs even without NVLink, just using the PCIe for the transfers. Depending on the model the performance is sometimes not all that different. I believe for solely inference on some models the speed difference may barely be noticeable, where for other training activities it may make 10+% difference [1] [0] https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html#single-machine-model-parallel-best-practices https://pytorch.org/tutorials/intermediate/model_parallel_tu... [1] https://huggingface.co/transformers/v4.9.2/performance.html https://huggingface.co/transformers/v4.9.2/performance.html
- bick_nyers 4y agoI remember reading about a guy who soldered 2GB VRAM modules on his 3060 12GB (replacing the 1GB modules) and was able to attain 24GB on that card. Or something along those lines.