3 ms·

I was talking about the new open models with a group of people yesterday, and saying how good they're getting. The big question is: Can any company now compete

by pyman 1y ago

I was talking about the new open models with a group of people yesterday, and saying how good they're getting. The big question is: Can any company now compete with the big players? Or even more interesting, like you showed in your research, are proprietary models becoming less relevant now that anyone can run these models locally? This trend of better open models that run locally is really picking up. Do you think we'll get to a point where we won't need to buy AI tokens anymore?

simonw 1y ago
The problem is cost. A machine that can run a decent local model costs thousands of dollars to buy and won't produce results as good as a model that runs on $30,000+ dedicated servers. Meanwhile you can rent access to LLMs running on those expensive machines for fractions of a cent (because you are sharing them with thousands of other users). I don't think cost will be a reason to use local models for a very long time, if ever.