3 ms·
There is a coherent argument that once LLMs reach the top of their S curve, the gap between small/medium local models and large cloud hosted ones converges. Es
by spacebanana7 14d ago
There is a coherent argument that once LLMs reach the top of their S curve, the gap between small/medium local models and large cloud hosted ones converges.
Especially if GPU performance increases or market oversupply mean you can get good performance for a couple thousand dollars.
I’m not sure about the nature or timeframe for an S curve in LLMs but I don’t think it’s unreasonable to think about one, nor to entertain the hosting consequences of a progression on one.
- TeMPOraL 14d agoI don't know the argument so I won't insist on the point, but I fail to see how it is relevant. Even if all proprietary LLMs disappeared today, efficiencies of scale alone mean the big cloud vendors can take the same open-weight LLMs you use locally, and sell inference with them for less money, and much more reliably, than you can afford yourself.
- spacebanana7 14d agoLocal machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?
- TeMPOraL 14d agoIf you have a machine and suddenly realize you can run LLMs on it, yes. If you're buying a machine specifically so it's capable of running LLMs for you, then the purchase cost is your up-front payment for the inference you'll run. And between that and electricity costs, cloud has you beat.
- DANmode 14d agoEvery Apple machine ships with LLMs. So, that’s a decent amount of people who could realize it today! Apple will be leaning into that further. No other play makes sense.
- layer8 14d agoMost people don’t have local machines with sufficient computational power and RAM.
- fragmede 14d agosufficient CPU and RAM can be found in a smartphone though, so it depends on which model you're talking about.
- layer8 14d agoWe were talking about the general case for personal use. What is possible on smartphones now with local models is not comparable to what people are using ChatGPT and friends for, and that will remain the case for the foreseeable future.
- DANmode 13d agoAre you sure? I see LOTS of text transformation tasks…