5 ms·
If you have a lot of money (but not H100/A100 money), get 4090s as they're currently the best bang for your buck on the CUDA side (according to George Hotz). If
by spikedoanz 3y ago
If you have a lot of money (but not H100/A100 money), get 4090s as they're currently the best bang for your buck on the CUDA side (according to George Hotz). If broke, get multiple second hand 3090s. https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni.... If unwilling to spend any money at all and just want to play around with llama70b, look into petals https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals
- flangola7 3y ago4090 has the same amount of VRAM, which is the important factor.
- spikedoanz 3y agoright. I opted for a used 3090 myself and plan to get a 2nd one soon. At current market prices 2x 3090s is cheaper than a single 4090 and provides double vram with more performance. If fine-tuning/lora-ing and energy efficiency is a concern though, I would opt for a 4090 since it is both far faster and far more efficient
- XTXinverseXTY 3y ago> according to George Hotz In this video from June, George Hotz says to go with "3090s over 4090s. 3090s have NVLink... 4090s are $1600 and 3090s are 750. RAM bandwidth is about the same." Has he changed his recommendations since then? https://youtu.be/Mr0rWJhv9jU?t=1157 https://youtu.be/Mr0rWJhv9jU?t=1157