5 ms·

~~This is not the direction I was hoping Apple would go with AI. With all the neural this and that bits baked into apple silicon, it has seemed [0] for a while

~~This is not the direction I was hoping Apple would go with AI. With all the neural this and that bits baked into apple silicon, it has seemed [0] for a while that Apple wanted to run all these workloads locally, but this partnership seems like a significant privacy backslide. Another comment in this thread said something about they’re using b Apple silicon for these workloads, but didn’t give an indication of whether that silicon lives in Apple datacenters or OpenAI ones. [0] https://news.ycombinator.com/item?id=38725167~~ https://news.ycombinator.com/item?id=38725167~~ edit: I should have mentioned that I didn’t have a chance to watch the video yet; a reply to my comment mentioned that it’s addressed in the video so I’ll go watch that later

astrange 2y ago
Watch the keynote, it's all clearly explained and you don't need to learn about it from HN comments.
- saagarjha 2y ago
  As I'm sure you're aware people who read Hacker News comments are generally unlikely to watch a hour+ keynote.
- philsnow 2y ago
  I should have put a disclaimer saying that I hadn’t had a chance to watch the video yet. Thanks for mentioning that it’s addressed, I’ll take a look later.
noahtallen 2y ago
I don't think this is a fair take. It sounds like the vast majority of the new AI features (including the local personal context for Siri, the various text/image editing features, better photo categorization, and the list goes on) are all local, on-device models, which can, if needed, use Apple's private cloud. That requires public researcher verification of server software for iOS to even talk to it. (Allegedly :)) The OpenAI partnership is seemingly only if Siri decides it doesn't have the full context needed to answer. (E.g. if you ask something very creative/generative.) At that point, Siri says "hey, chatGPT might be better at answering this, do you consent to me sending your prompt to them?" and then you get to choose. Apple's partnership also seems to include the various settings that prevent OpenAI from tracking/training on the prompts sent in. Honestly, that more creative side of genAI is not as interesting in the full context of Apple Intelligence. The real power is coming from the local, personal context, where Siri can deduce context based on your emails, messages, calendar events, maps, photos, etc. to really deeply integrate with apps. (Allegedly!) And that part is not OpenAI.
- atlex2 2y ago
  Agreed. Apple pretty clearly focused on building an action-tuned model. Also, notice how in the videos you barely see any "Siri speech". I wonder what they used for pre-training, but probably they did it with much more legit datasources-- They're launching with English only.
Workaccount2 2y ago
Apple is in the position where it caters primarily to the tech ignorant, so coming out an explaining that Apple LLM is a bit worse (read: far worse) than the cool LLM's on the internet because they are privacy conscious is a non-starter. Local LLM's on regular local hardware (i.e. no $500+ dedicated GPUs) is way far behind SoTA models right now. Apple is not gonna be in a position where you can practically real-time intelligently chat with Android phones while iPhones are churning out 3 tokens/second of near useless babbling.
- philsnow 2y ago
  (I haven’t watched the video yet) I completely agree about the market positioning and not keeping up with other platforms’ abilities being a non-starter. I just hope it will be clear how to keep my external brain (phone) from being scanned by OpenAI. (I don’t want it to seem like I’m just a hater of either Apple or OpenAI; I’m a more-recent adopter of Apple tech and I’m not looking back, and I have an OpenAI subscription and find it invaluable for some uses.) Another thing I’m going to be looking for in the video is how this initiative jibes with the greenness Apple has been passing really hard. If they’re bringing this kind of generative AI from niches to every iphone, it seems that would add a fair amount of power consumption.
  - noahtallen 2y ago
    > I just hope it will be clear how to keep my external brain (phone) from being scanned by OpenAI. It's very clear, the keynote demonstrates that Siri passing a prompt to chatGPT is completely opt-in and only happens when Siri thinks the prompt needs the more generative/creative model that OpenAI provides.

astrange 2y ago

Watch the keynote, it's all clearly explained and you don't need to learn about it from HN comments.

saagarjha 2y ago
As I'm sure you're aware people who read Hacker News comments are generally unlikely to watch a hour+ keynote.
philsnow 2y ago
I should have put a disclaimer saying that I hadn’t had a chance to watch the video yet. Thanks for mentioning that it’s addressed, I’ll take a look later.

noahtallen 2y ago

I don't think this is a fair take. It sounds like the vast majority of the new AI features (including the local personal context for Siri, the various text/image editing features, better photo categorization, and the list goes on) are all local, on-device models, which can, if needed, use Apple's private cloud. That requires public researcher verification of server software for iOS to even talk to it. (Allegedly :)) The OpenAI partnership is seemingly only if Siri decides it doesn't have the full context needed to answer. (E.g. if you ask something very creative/generative.) At that point, Siri says "hey, chatGPT might be better at answering this, do you consent to me sending your prompt to them?" and then you get to choose. Apple's partnership also seems to include the various settings that prevent OpenAI from tracking/training on the prompts sent in. Honestly, that more creative side of genAI is not as interesting in the full context of Apple Intelligence. The real power is coming from the local, personal context, where Siri can deduce context based on your emails, messages, calendar events, maps, photos, etc. to really deeply integrate with apps. (Allegedly!) And that part is not OpenAI.

atlex2 2y ago
Agreed. Apple pretty clearly focused on building an action-tuned model. Also, notice how in the videos you barely see any "Siri speech". I wonder what they used for pre-training, but probably they did it with much more legit datasources-- They're launching with English only.

Workaccount2 2y ago

Apple is in the position where it caters primarily to the tech ignorant, so coming out an explaining that Apple LLM is a bit worse (read: far worse) than the cool LLM's on the internet because they are privacy conscious is a non-starter. Local LLM's on regular local hardware (i.e. no $500+ dedicated GPUs) is way far behind SoTA models right now. Apple is not gonna be in a position where you can practically real-time intelligently chat with Android phones while iPhones are churning out 3 tokens/second of near useless babbling.

philsnow 2y ago
(I haven’t watched the video yet) I completely agree about the market positioning and not keeping up with other platforms’ abilities being a non-starter. I just hope it will be clear how to keep my external brain (phone) from being scanned by OpenAI. (I don’t want it to seem like I’m just a hater of either Apple or OpenAI; I’m a more-recent adopter of Apple tech and I’m not looking back, and I have an OpenAI subscription and find it invaluable for some uses.) Another thing I’m going to be looking for in the video is how this initiative jibes with the greenness Apple has been passing really hard. If they’re bringing this kind of generative AI from niches to every iphone, it seems that would add a fair amount of power consumption.
- noahtallen 2y ago
  > I just hope it will be clear how to keep my external brain (phone) from being scanned by OpenAI. It's very clear, the keynote demonstrates that Siri passing a prompt to chatGPT is completely opt-in and only happens when Siri thinks the prompt needs the more generative/creative model that OpenAI provides.