4 ms·
It's very convenient for Apple to do this: less expenses on costly AI chips, and more excuses to ask customers to buy their latest hardware.
by coliveira 1y ago
It's very convenient for Apple to do this: less expenses on costly AI chips, and more excuses to ask customers to buy their latest hardware.
- nine_k 1y agoUsers have to pay for the compute somehow. Maybe by paying for models run in datacenters. Maybe paying for hardware that's capable enough to run models locally.
- Bootvis 1y agoI can upgrade to a bigger LLM I use through an API with one click. If it runs on my device device I need to buy a new phone.
- nine_k 1y agoI* can run the model on my device, no matter if I have an internet connection, nor if I have a permission from whoever controls the datacenter. I can run the model against highly private data while being certain that the private data never leaves my device. It's a different set of trade-offs. * Theoretically; I don't own an iPhone.
- eru 1y agoWell, unless it's open source, you can't be so certain. But more certain than when processing in the cloud, that's true.
- lostlogin 1y agoBut also: if Apple's way works, it’s incredibly wasteful. Server side means shared resources, shared upgrades and shared costs. The privacy aspect matters, but at what cost?
- shakna 1y agoServer side means an excuse to not improve model handling everywhere you can, and increasing global power usage by noticable percentage point, at a time when we're approaching "point of no return" with burning out the only planet we can live on. The cost, so far, is greater.
- hu3 1y ago> Server side means an excuse to not improve model handling everywhere you can... How so if efficiency is key for datacenters to be competitive? If anything it's the other way around.
- coliveira 1y agoThe previous commenter is right in that server-side companies have little incentive to do less, especially when they're backed by investors money. Client-side AI will be bound by device capabilities and customer investment in new devices.
- shakna 1y agoOr, instead of improving efficiency, they go ahead and just deploy more generators [0]. Stop gap measures are cheaper. [0] https://interestingengineering.com/innovation/elon-musk-xai-unauthorized-gas-turbines-memphis https://interestingengineering.com/innovation/elon-musk-xai-...
- eru 1y agoWell, if it were easier to build power stations, they'd do so.
- thfuran 1y agoMore like squinting to see if it's still visible in the rear view mirror.
- eru 1y agoHow does running AI workloads on end user devices magically make them use less energy?
- zamadatix 1y agoIf iPhones were the efficient/smart way to pay for compute then Apple's datacenter would be built with those instead of servers.