3 ms·
I don't know the argument so I won't insist on the point, but I fail to see how it is relevant. Even if all proprietary LLMs disappeared today, efficiencies of
by TeMPOraL 13d ago
I don't know the argument so I won't insist on the point, but I fail to see how it is relevant. Even if all proprietary LLMs disappeared today, efficiencies of scale alone mean the big cloud vendors can take the same open-weight LLMs you use locally, and sell inference with them for less money, and much more reliably, than you can afford yourself.
- spacebanana7 13d agoLocal machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?
- TeMPOraL 13d agoIf you have a machine and suddenly realize you can run LLMs on it, yes. If you're buying a machine specifically so it's capable of running LLMs for you, then the purchase cost is your up-front payment for the inference you'll run. And between that and electricity costs, cloud has you beat.
- DANmode 13d agoEvery Apple machine ships with LLMs. So, that’s a decent amount of people who could realize it today! Apple will be leaning into that further. No other play makes sense.
- layer8 13d agoMost people don’t have local machines with sufficient computational power and RAM.
- fragmede 13d agosufficient CPU and RAM can be found in a smartphone though, so it depends on which model you're talking about.
- layer8 13d agoWe were talking about the general case for personal use. What is possible on smartphones now with local models is not comparable to what people are using ChatGPT and friends for, and that will remain the case for the foreseeable future.
- DANmode 12d agoAre you sure? I see LOTS of text transformation tasks…