3 ms·
I thought it was possible to use two cards and “share” the ram? Then it would still be a good deal. But maybe I’m wrong.
by phero_cnstrcts 3y ago
I thought it was possible to use two cards and “share” the ram? Then it would still be a good deal. But maybe I’m wrong.
- samspenc 3y agoThis is only supported with the previous generation NVidia 3090, it is apparently possible to combine two 3090s with 24 GB VRAM and 'fuse' them with NVLink to act as a single high-powered GPU with 48 GB VRAM combined. NVidia no longer supports this for the 40-series, I think this is because they want anyone interested in using their GPUs for LLMs to buy the pricier models with more VRAM.
- two_in_one 3y agoTheoretically you can use as many as GPUs you want in parallel. LLMs are easy to split and run in model parallel configuration (for big models which don't fit on one card). or data parallel for performance, when the same model runs different batches on GPUs. PyTorch has full support for both modes, afaik.