4 ms·
You can build an x86 machine that can fully run DeepSeek R1 with 512GB VRAM for ~$2,500?
by smith7018 2y ago
You can build an x86 machine that can fully run DeepSeek R1 with 512GB VRAM for ~$2,500?
- deleted 2y ago[deleted]
- ta988 2y agoYou will have to explain to me how.
- hbbio 2y agoHow would you compare the tok/sec between this setup and the M3 Max?
- deleted 2y ago[deleted]
- aurareturn 2y ago3.5 - 4.5 tokens/s on the $2,000 AMD Epyc setup. Deepseek 671b q4. The AMD Epyc build is severely bandwidth and compute constrained. ~40 tokens/s on M3 Ultra 512GB by my calculation.
- sgt 2y agoWhat kind of Nvidia-based rig would one need to achieve 40 tokens/sec on Deepseek 671b? And how much would it cost?
- aurareturn 2y agoAround 5x Nvidia A100 80GB can fit 671b Q4. $50k just for the GPUs and likely much more when including cooling, power, motherboard, CPU, system RAM, etc.
- sgt 2y agoSo the M3 Ultra is amazing value then. And from what I could tell, an equivalent AMD Epyc would still be so constrained that we're talking 4-5 tokens/s. Is this a fair assumption?
- Aeolun 2y agoThe Epyc would only set you back $2000 though, so it’s only a slightly worse price/return.
- adgjlsfhk1 2y agoNo. The advantage of Epic is you get 12 channels of ram so it should be ~6x faster than a consumer cpu.
- 2y ago