3 ms·
I built a DIY PC with used GPUs (2x RTX 3090) for around 2300€ earlier this year. You can probably do it for slightly less now (i also added 128GB RAM and NVLin
by Tepix 3y ago
I built a DIY PC with used GPUs (2x RTX 3090) for around 2300€ earlier this year. You can probably do it for slightly less now (i also added 128GB RAM and NVLink). You can generate text with >10 tok/s with that setup.
Make sure to get a PSU with more than 1000W.
Air cooling is a challenge, but it's possible.
Recommended reading: Tim Dettmer's guide https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
- byteknight 3y agoYou can't even buy the graphics cards for that price where the heck did you find that?
- Tepix 3y agoAlmost everything was used, the GPUs were around 720€ each. You can now buy them as low as 600€. Make sure to get two identical ones if you plan to connect them with NVLink.