4 ms·
Maybe at some point, Apple is/was trying to do everything locally but it appears they have recently decided to move away from that idea and use OpenAI. I can u
by stingraycharles 1y ago
Maybe at some point, Apple is/was trying to do everything locally but it appears they have recently decided to move away from that idea and use OpenAI.
I can understand why: you’re only using locally-run AI models every so often (maybe a few times a day), but when you use it, you still want it to be fast.
So it will need to be a pretty heavy AI chip in your phone to be able to deliver that, which spends most of the time idling.
Since compute costs are insane for AI, it only makes sense to optimize this and do the inference in the cloud.
Maybe at some point local AI will be possible, but they’ll always be able to run much more powerful models in the cloud, because it makes much more sense from an economics point of view.
- jpalawaga 1y agoGoogle also has AI models optimized to run on phones, they're just in a lot better of a position to actually build purpose-built LLMs for phones. It's not clear to me why certain classes of things still end up farmed out to the cloud (such as this, or is it?). Maybe their LLM hasn't been built in a very pluggable fashion.
- Hizonner 1y ago> they have recently decided to move away from that idea and use OpenAI. ... although, to be fair, they're negotiating with OpenAI to run the models in "secure enclaves", which should, assuming everything works right which is a huge assumption, keep Apple or anybody else from reaching inside and seeing what the model is "thinking about".