3 ms·
The gap between local models and SOTA is around 6 months and it's either steady or dropping. (Obviously this depends on your benchmark and preferences.)
by wmf 1y ago
The gap between local models and SOTA is around 6 months and it's either steady or dropping. (Obviously this depends on your benchmark and preferences.)
- criddell 1y agoSeriously? So I can run the best models from 2024 at home now? For example, what would I need to run Open AI's o1 model from 2024 at home? Are there good guides for setting this up?
- wmf 1y agoIt's not the same model, but for example GPT-OSS-120B is smarter than o1. The guide is buy 128 GB of VRAM then install LM Studio.
- criddell 1y agoAn NVIDIA 5090 with 128 GB of VRAM is $13k. It doesn’t make any sense to run that at home when you can pay OpenAI $20 / month to use it (it would take more than 50 years to spend $13k at OpenAI this way). So technically you might be able to run a six month old model at home, but it would be foolish to do so from a financial point of view. Or is there a way to get 128 GB of VRAM for a lot less than that?
- wmf 1y agoRyzen AI Max is $2,000, M4 Max is $3,500, and DGX Spark is $4,000. Still not really economically feasible but I see it as an insurance policy. And that's the most expensive local model; smaller models will run on any PC.