6 ms·
If you can fit the training into 24GB, a used RTX 3090 for $700-$800 seems like a good deal at the moment. They are about 45-65% as fast as the A100 according t
by Tepix 4y ago
If you can fit the training into 24GB, a used RTX 3090 for $700-$800 seems like a good deal at the moment. They are about 45-65% as fast as the A100 according to https://bizon-tech.com/gpu-benchmarks/NVIDIA-RTX-3090-vs-NVIDIA-A100-40-GB-(PCIe)/579vs592 https://bizon-tech.com/gpu-benchmarks/NVIDIA-RTX-3090-vs-NVI...
So if you buy two of these cards it will take 12-13 days instead of 38 hours but only require a $2500 PC.
James Betker, who created tortoise TTS, built his own $15k machine with 8x RTX 3090 and trained the models with it. He now works for OpenAI…
- klaudioz 4y agoAny link to the 15k machine ?. Maybe it is cheaper now.
- Tepix 4y agoI think it was a DIY machine, those RTX 3090 have gotten cheaper for sure. From my experience, going beyond 4 GPUs is a pricey affair. See [§]. All but one model of the RTX3090 require at least 3 slots. If 4 GPUs connected via PCIe 4.0x16 are enough you can choose among various sRTX4 boards for 3000 series AMD Threadripper CPUs. [§] https://www.reddit.com/r/deeplearning/comments/tw0olq/comment/i3cuyks/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/deeplearning/comments/tw0olq/commen... Another useful URL: https://www.pugetsystems.com/labs/articles/Quad-GeForce-RTX-3090-in-a-desktop---Does-it-work-1935/ https://www.pugetsystems.com/labs/articles/Quad-GeForce-RTX-...
- Tepix 4y agoRecommended reading: https://timdettmers.com/2023/01/16/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/16/which-gpu-for-deep-learni... TL;DR: You probably don't need that expensive Threadripper because 2x PCIe 4.0 x16 will not be very beneficial. Go cheap, go 2x PCIe 4.0 x8.