2 ms·
The problem is cost. A machine that can run a decent local model costs thousands of dollars to buy and won't produce results as good as a model that runs on $30
by simonw 1y ago
The problem is cost. A machine that can run a decent local model costs thousands of dollars to buy and won't produce results as good as a model that runs on $30,000+ dedicated servers. Meanwhile you can rent access to LLMs running on those expensive machines for fractions of a cent (because you are sharing them with thousands of other users).
I don't think cost will be a reason to use local models for a very long time, if ever.