3 ms·

Llama-scale models should have no problem running on iPhone. I doubt it will be a unique feature, though - not only is the Neural Engine near-useless for infere

Llama-scale models should have no problem running on iPhone. I doubt it will be a unique feature, though - not only is the Neural Engine near-useless for inferencing, Android devices have a much better runtime for loading and managing large models. Comparing Apple's Pytorch contributions to what Microsoft is doing on ARM with ONNX, it (ironically) feels like non-Apple platforms are teed-up better for local AI. Running LLaMA on an Ampere server or Rockchip SOC is easy as pie. If Apple does go this route, I feel like they're setting themselves up for disruption. Someone else (hell, maybe even Meta) will do it better, and Apple's implementation will hold on by the thread of native integration it uses. It's not a bad or new situation, but I'm going to bet that Apple will hamstring themselves by locking competitors out. Especially if the current pace of model development keeps up.

desro 4y ago
If I had to place a wager I'd guess that it'll be pretty typical Apple: they won't be first, but the ironclad integration of their hardware and software will enable some unique stuff. I'm picturing what Microsoft recently demoed with Office 365 Co-Pilot [0] but billed more as for your entire life beyond work. I have to think they wouldn't mobilize some of their capital reserves if need be in order to avoid missing this wave. Part of me thinks that's the only reasonable cause for Siri being so terribly useless for so long— that it's because they have a "leapfrog" up their sleeves. I acknowledge this is a pretty heavy cope though. [0] https://web.archive.org/web/20230420002004/https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/ https://web.archive.org/web/20230420002004/https://blogs.mic...