4 ms·

I can only be happy for this. I can't really judge at this point, but I really doubt there is hardware limitation for that, e.g. iPhone 15 Pro isn't so much po

by _8nbg 2y ago

I can only be happy for this. I can't really judge at this point, but I really doubt there is hardware limitation for that, e.g. iPhone 15 Pro isn't so much powerful than iPhone 15. Even when it is - they could utilize their server side AI solution, but for them greed comes first.

jhugo 2y ago
Given Apple’s general approach, I suspect it’s on-device and these are the only devices that can currently achieve a good experience. I’d like to see them make the devices work together - having an iPhone, Mac or iPad run the model but access it via a watch or HomePod.
bzzzt 2y ago
The local AI model Apple is using needs at least 8Gb. (I guess not all for the AI, but the rest of the phone also needs memory to function)
simonw 2y ago
6GB RAM on the iPhone 15, 8GB on the iPhone 15 Pro. That 2GB really could make the difference: they have a 3B language model, which is going to be memory hungry. There needs to be enough memory left for the rest of the phone and apps to function.
- _8nbg 2y ago
  > 6GB RAM on the iPhone 15, 8GB on the iPhone 15 Pro. > That 2GB really could make the difference: they have a 3B language model, which is going to be memory hungry. Even when it is - they could utilize their server side AI solution, but for them greed comes first.
  - flappyeagle 2y ago
    What is this comment even? Greed comes first? Apple likes money? You were saying above that you’re glad you don’t have it and now you’re complaining that you don’t have it?
- htrp 2y ago
  This! And a reminder that Phi1.5 comes in at 2.8GB. Getting a performant model even while using aggressive quantization is going to require more RAM. iPhone 16 ProAI may end up having notebook levels of ram (12 / 16GB)
harrygeez 2y ago
For on-device AI as a matter of fact it is. 35 TOPS vs 17 TOPS is quite substantial. For reference Microsoft demands 40+ TOPS for “Copilot+” PCs. Probably they could offload more to their servers but it wouldn’t be the same experience definitely.
_8nbg 2y ago
[flagged]
- supermatt 2y ago
  What double standard are you talking about? Are you suggesting that you have to pay for recent hardware for local privacy otherwise you use ChatGPT and its not private? As many have already pointed out to you, its most likely the earlier hardware simply wont run their local model. The partnership with OpenAI includes an agreement that they will not track/log apple users, and even then it is a fallback option (after apples OWN cloud model) that users must specifically agree to.
- aikinai 2y ago
  You haven't watched or read what they announced then. Almost all queries are processed locally or in the "Private Cloud" on Apple chips in Apple data centers. For queries that are better answered by ChatGPT, the phone will explicitly ask you if you want to forward it to ChatGPT. "Bring ChatGPT to Siri" is extremely misleading. It's less integrated with Siri than Google is.
- seanhunter 2y ago
  Calling people who disagree with you "fanboys" doesn't help elevate the conversation. That said, whether you think me a fanboy or not, your openAI criticism is wide of the mark. The chatgpt/openai integration is apparently going to be 100% opt-in and you will be prompted for consent every time data leaves your device. So much so that the economist said it risks becoming "ChatGDPR"[1]. [1] https://www.economist.com/business/2024/06/11/hey-siri-help-me-get-apple-out-of-an-ai-shaped-hole https://www.economist.com/business/2024/06/11/hey-siri-help-... (requires a subscription)

by _8nbg 2y ago

jhugo 2y ago

Given Apple’s general approach, I suspect it’s on-device and these are the only devices that can currently achieve a good experience. I’d like to see them make the devices work together - having an iPhone, Mac or iPad run the model but access it via a watch or HomePod.

bzzzt 2y ago

The local AI model Apple is using needs at least 8Gb. (I guess not all for the AI, but the rest of the phone also needs memory to function)

simonw 2y ago

6GB RAM on the iPhone 15, 8GB on the iPhone 15 Pro. That 2GB really could make the difference: they have a 3B language model, which is going to be memory hungry. There needs to be enough memory left for the rest of the phone and apps to function.

_8nbg 2y ago
> 6GB RAM on the iPhone 15, 8GB on the iPhone 15 Pro. > That 2GB really could make the difference: they have a 3B language model, which is going to be memory hungry. Even when it is - they could utilize their server side AI solution, but for them greed comes first.
- flappyeagle 2y ago
  What is this comment even? Greed comes first? Apple likes money? You were saying above that you’re glad you don’t have it and now you’re complaining that you don’t have it?
htrp 2y ago
This! And a reminder that Phi1.5 comes in at 2.8GB. Getting a performant model even while using aggressive quantization is going to require more RAM. iPhone 16 ProAI may end up having notebook levels of ram (12 / 16GB)

harrygeez 2y ago

For on-device AI as a matter of fact it is. 35 TOPS vs 17 TOPS is quite substantial. For reference Microsoft demands 40+ TOPS for “Copilot+” PCs. Probably they could offload more to their servers but it wouldn’t be the same experience definitely.

_8nbg 2y ago

[flagged]

supermatt 2y ago
What double standard are you talking about? Are you suggesting that you have to pay for recent hardware for local privacy otherwise you use ChatGPT and its not private? As many have already pointed out to you, its most likely the earlier hardware simply wont run their local model. The partnership with OpenAI includes an agreement that they will not track/log apple users, and even then it is a fallback option (after apples OWN cloud model) that users must specifically agree to.
aikinai 2y ago
You haven't watched or read what they announced then. Almost all queries are processed locally or in the "Private Cloud" on Apple chips in Apple data centers. For queries that are better answered by ChatGPT, the phone will explicitly ask you if you want to forward it to ChatGPT. "Bring ChatGPT to Siri" is extremely misleading. It's less integrated with Siri than Google is.
seanhunter 2y ago
Calling people who disagree with you "fanboys" doesn't help elevate the conversation. That said, whether you think me a fanboy or not, your openAI criticism is wide of the mark. The chatgpt/openai integration is apparently going to be 100% opt-in and you will be prompted for consent every time data leaves your device. So much so that the economist said it risks becoming "ChatGDPR"[1]. [1] https://www.economist.com/business/2024/06/11/hey-siri-help-me-get-apple-out-of-an-ai-shaped-hole https://www.economist.com/business/2024/06/11/hey-siri-help-... (requires a subscription)