4 ms·

After reading through all the comments, I'm surprised that no one has looked at this from a Siri-centric perspective [except for the comment about Clippy ;)].

by desro 3y ago

After reading through all the comments, I'm surprised that no one has looked at this from a Siri-centric perspective [except for the comment about Clippy ;)]. To me this looks like laying the groundwork for centralizing the most relevant and important information about yourself, which so crucially includes the contexts you are in! and are so often painstaking to actually record in detail when journalling manually. For what purpose? I genuinely think this leads to a Siri that's finally useful in a way that leapfrogs everything and everyone else out there. Think GPT-4(+) with an insane amount of detail and context tailored to you, and (hopefully!) executed in a way that re-affirms Apple's stated commitment to privacy and security. I have only the most cursory understanding of ML technology but have obviously been trying to follow along with everything pretty much since DALLE-2 made such a splash. With some of the impressive performance shown in smaller models, sometimes with different quantization, I think that Apple Neural Engine silicon might be getting more attention soon... okay, "soon" is probably over-optimistic. But – in September '22 I asked online about running Stable Diffusion[0] on M1 chips on iPad, since I was able to run it on an M1 Mac Mini. The 8-ball said "outlook not so good," and yet by November 8th liuliu had it running on _iPhone_[1]. These are truly interesting times. [0] I know this is not a 1:1 comparison. But Llama ran on my MacBook without any CoreML optimization. Maybe there will be a tier of requests that could be handled by a smaller model on-device, and more complex stuff heads to the datacenter. I am an amateur at best; don't listen to me. [1] https://liuliu.me/eyes/stretch-iphone-to-its-limit-a-2gib-model-that-can-draw-everything-in-your-pocket/ https://liuliu.me/eyes/stretch-iphone-to-its-limit-a-2gib-mo...

smoldesu 3y ago
Llama-scale models should have no problem running on iPhone. I doubt it will be a unique feature, though - not only is the Neural Engine near-useless for inferencing, Android devices have a much better runtime for loading and managing large models. Comparing Apple's Pytorch contributions to what Microsoft is doing on ARM with ONNX, it (ironically) feels like non-Apple platforms are teed-up better for local AI. Running LLaMA on an Ampere server or Rockchip SOC is easy as pie. If Apple does go this route, I feel like they're setting themselves up for disruption. Someone else (hell, maybe even Meta) will do it better, and Apple's implementation will hold on by the thread of native integration it uses. It's not a bad or new situation, but I'm going to bet that Apple will hamstring themselves by locking competitors out. Especially if the current pace of model development keeps up.
- desro 3y ago
  If I had to place a wager I'd guess that it'll be pretty typical Apple: they won't be first, but the ironclad integration of their hardware and software will enable some unique stuff. I'm picturing what Microsoft recently demoed with Office 365 Co-Pilot [0] but billed more as for your entire life beyond work. I have to think they wouldn't mobilize some of their capital reserves if need be in order to avoid missing this wave. Part of me thinks that's the only reasonable cause for Siri being so terribly useless for so long— that it's because they have a "leapfrog" up their sleeves. I acknowledge this is a pretty heavy cope though. [0] https://web.archive.org/web/20230420002004/https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/ https://web.archive.org/web/20230420002004/https://blogs.mic...

by desro 3y ago

smoldesu 3y ago

Llama-scale models should have no problem running on iPhone. I doubt it will be a unique feature, though - not only is the Neural Engine near-useless for inferencing, Android devices have a much better runtime for loading and managing large models. Comparing Apple's Pytorch contributions to what Microsoft is doing on ARM with ONNX, it (ironically) feels like non-Apple platforms are teed-up better for local AI. Running LLaMA on an Ampere server or Rockchip SOC is easy as pie. If Apple does go this route, I feel like they're setting themselves up for disruption. Someone else (hell, maybe even Meta) will do it better, and Apple's implementation will hold on by the thread of native integration it uses. It's not a bad or new situation, but I'm going to bet that Apple will hamstring themselves by locking competitors out. Especially if the current pace of model development keeps up.

desro 3y ago
If I had to place a wager I'd guess that it'll be pretty typical Apple: they won't be first, but the ironclad integration of their hardware and software will enable some unique stuff. I'm picturing what Microsoft recently demoed with Office 365 Co-Pilot [0] but billed more as for your entire life beyond work. I have to think they wouldn't mobilize some of their capital reserves if need be in order to avoid missing this wave. Part of me thinks that's the only reasonable cause for Siri being so terribly useless for so long— that it's because they have a "leapfrog" up their sleeves. I acknowledge this is a pretty heavy cope though. [0] https://web.archive.org/web/20230420002004/https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/ https://web.archive.org/web/20230420002004/https://blogs.mic...