3 ms·
That IS me. $5K is not enough to run an LLM at home (beyond the non-functional reduced quantization smaller models).
by koheripbal 4y ago
That IS me. $5K is not enough to run an LLM at home (beyond the non-functional reduced quantization smaller models).
- bick_nyers 4y agoAhh yes, looks like I was too generous with my numbers, the new Quadro with 48GB VRAM is $7k, so you probably would need $14k and a Threadripper/Xeon/EPYC workstation because you won't have enough PCIE lanes/RAM/Memory Bandwidth otherwise. So maybe more accurate is $200k+ a year and $20-30k on a workstation. I grew up on $20k a year, the numbers in tech. are baffling!