2 ms·
Local machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?
by spacebanana7 6d ago
Local machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?
- TeMPOraL 6d agoIf you have a machine and suddenly realize you can run LLMs on it, yes. If you're buying a machine specifically so it's capable of running LLMs for you, then the purchase cost is your up-front payment for the inference you'll run. And between that and electricity costs, cloud has you beat.
- DANmode 6d agoEvery Apple machine ships with LLMs. So, that’s a decent amount of people who could realize it today! Apple will be leaning into that further. No other play makes sense.
- layer8 6d agoMost people don’t have local machines with sufficient computational power and RAM.
- fragmede 6d agosufficient CPU and RAM can be found in a smartphone though, so it depends on which model you're talking about.
- layer8 6d agoWe were talking about the general case for personal use. What is possible on smartphones now with local models is not comparable to what people are using ChatGPT and friends for, and that will remain the case for the foreseeable future.
- DANmode 5d agoAre you sure? I see LOTS of text transformation tasks…