3 ms·
Local machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?
by spacebanana7 2mo ago
Local machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?
- TeMPOraL 2mo agoIf you have a machine and suddenly realize you can run LLMs on it, yes. If you're buying a machine specifically so it's capable of running LLMs for you, then the purchase cost is your up-front payment for the inference you'll run. And between that and electricity costs, cloud has you beat.
- DANmode 2mo agoEvery Apple machine ships with LLMs. So, that’s a decent amount of people who could realize it today! Apple will be leaning into that further. No other play makes sense.
- layer8 2mo agoMost people don’t have local machines with sufficient computational power and RAM.
- fragmede 2mo agosufficient CPU and RAM can be found in a smartphone though, so it depends on which model you're talking about.
- layer8 2mo agoWe were talking about the general case for personal use. What is possible on smartphones now with local models is not comparable to what people are using ChatGPT and friends for, and that will remain the case for the foreseeable future.
- DANmode 2mo agoAre you sure? I see LOTS of text transformation tasks…