4 ms·
You can get a second hand RTX 3090 with $800. So in theory you could build a cluster with less than $40k for all hardware. Not that this is pocket change, but g
by elorant 3y ago
You can get a second hand RTX 3090 with $800. So in theory you could build a cluster with less than $40k for all hardware. Not that this is pocket change, but given the capabilities of such a model the expense doesn’t sound that much.
- rthnbgrredf 3y agoOr one can try to rent a cluster in the clouds.
- karmasimida 3y agoyou don't have the nvlink, how would this work then
- elorant 3y agoYou don’t need the entire VRAM to be accessed as one. Everything is parallelized by the framework. You lose some speed, but it’s the only way to do it. There’s no freaking way to have 800 GB of unified VRAM.
- _zoltan_ 3y agoGH200 would like a word ;-)
- razodactyl 3y agoHahaha I love this.
- asfbnionio 3y agoAs I understand it, NVLink isn't actually required for much of anything. It allows P2P transfers and atomics. That same functionality can be used over PCIe on some GPUs with reduced performance. And the P2P features are not transparent and are lower-performance than staying inside one GPU, so generally software avoids P2P transfers even when they are available. The Nvidia documentation on this is absolutely horrible, the features vary wildly by model of GPU, and I claim no real expertise.
- htrp 3y ago3090's probably aren't gonna cut it because of the generational improvements in architecture (even if you could hack the drivers together to work)