7 ms·

So, if I've got this correct there's: 1. On-device AI 2. AI using Apple's servers 3. AI using ChatGPT/OpenAI's services (and others in the future) Number 1

So, if I've got this correct there's: 1. On-device AI 2. AI using Apple's servers 3. AI using ChatGPT/OpenAI's services (and others in the future) Number 1 will pass to number 2 if it thinks it requires the extra processing power, but number 3 will only be invoked with explicit user permission. [Edit: As pointed out below, other providers will be coming eventually.]

KaiserPro 2y ago
I don't know how they are going to square the privacy circle when at worst its a RAG based firehose to OpenAI, and at best you can just ask the model to leak your personal info.
sc077y 2y ago
I'm skeptical of the on-device AI. They crave edge compute but I'm doubtful their chips can handle a 7B param model. Maybe ironically with Microsoft's phi 3 mini 4k you can run this stuff on a cpu but today it's no where near good enough.
frenchie4111 2y ago
That was my sense as well. I would have appreciated some clarification on where the line between 1 and 2 was, although I am sure a YouTuber will deep dive on it as soon as they have it in their hands
gigel82 2y ago
The problem is they don't explicitly define when 1 can pass to 2 and whether we can fully and categorically disable it. As far as I know, 1 can pass to 2 when governments ask for some personal data or when Apple's ad model needs some intimate details for personalization.
- Tagbert 2y ago
  The information provided for level two is end to end encrypted and not stored so the risk level is pretty low here.
  - coolspot 2y ago
    Ent-to-end encrypted means that the other end (Apple/NSA) has access to it.
    - cyberpunk 2y ago
      Imagine the memory on their server is encrypted with an on-processor key (something like intel SGX) -- reading OS memory, e.g dumping from linux or hardware, you can't read it unless you somehow extract the key (which are different on each chip) from the physical chip. Now, the process running using that encrypted memory generates TLS keys for you to send the data, and operates on it only inside this secure enclave. There is no way to access it without destroying the chip, and even in this scenario it will be extremely expensive and imo unlikely, certainly impossible at scale. Some scientists may be able to do it once in a lab.
      - sumeno 2y ago
        Ok, I'm imagining. Now, is any of that actually true?
        sodality2 2y ago
        https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/
      - cyberpunk 2y ago
        BTW there is an entire industry popping up around exactly this sort of use case, it's called 'confidential computing' and CNCF have some software in the works (confidential containers iirc). I'm pretty excited to see what risc-v is going to bring to the party enclave wise.
    - Tagbert 2y ago
      It does need to process the data. The server has no persistent storage and no remote shell. It is a limited and locked down special-purpose iOS.
terramex 2y ago
Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hosted LLama 3 (or whatever will be released in next 6-12 months).
- hmottestad 2y ago
  Or Groq. They can do 1250 tokens/s with Llama 3 8B.
SoftTalker 2y ago
I see no real difference between 2 and 3. Once the data has left your device, it has left your device. There is no getting it back and you no longer have any control over it.
- r00fus 2y ago
  Hopefully they have some toggle in settings for this.
- solarkraft 2y ago
  Apple has demonstrated to be relatively trustworthy about privacy while most AI companies have demonstrated the opposite, so I do see a significant difference.
  - SoftTalker 2y ago
    Google was considered very cool and trustworthy at one point also. "Do no evil" and all that.
    - babypuncher 2y ago
      Google's entire buisness model was built on hoovering up and selling access to user data in the form of AdSense. Without that data, their business falls apart. Apple's business model is to entice people into a walled garden ecosystem where they buy lots of expensive hardware sold on high margins. They don't need user data to make this work, which is why they can more comfortably push features like end-to-end and no-knowledge encryption.
    - doctor_eval 2y ago
      Google was cool, once upon a time, but they always used your personal info pretty openly. The CEO a himself famously said, “The Google policy on a lot of things is to get right up to the creepy line and not cross it.” Apple has taken a markedly different approach, and has done so for years - E2E encryption, hashing and segmenting routes on maps, Secure Enclave, etc. While I think it’s perfectly reasonable to “trust no one”, and I fully agree that there may be things we don’t know, I don’t think there it’s reasonable to put Apple on the same (exceedingly low) level as Google.
spike021 2y ago
It sounded like 3 is meant for non-personal stuff. Basically like a search engine style feature. When you want to look up things like say sports records and info, or a movie and info about it, etc.

KaiserPro 2y ago

I don't know how they are going to square the privacy circle when at worst its a RAG based firehose to OpenAI, and at best you can just ask the model to leak your personal info.

sc077y 2y ago

I'm skeptical of the on-device AI. They crave edge compute but I'm doubtful their chips can handle a 7B param model. Maybe ironically with Microsoft's phi 3 mini 4k you can run this stuff on a cpu but today it's no where near good enough.

frenchie4111 2y ago

That was my sense as well. I would have appreciated some clarification on where the line between 1 and 2 was, although I am sure a YouTuber will deep dive on it as soon as they have it in their hands

gigel82 2y ago

The problem is they don't explicitly define when 1 can pass to 2 and whether we can fully and categorically disable it. As far as I know, 1 can pass to 2 when governments ask for some personal data or when Apple's ad model needs some intimate details for personalization.

Tagbert 2y ago
The information provided for level two is end to end encrypted and not stored so the risk level is pretty low here.
- coolspot 2y ago
  Ent-to-end encrypted means that the other end (Apple/NSA) has access to it.
  - cyberpunk 2y ago
    Imagine the memory on their server is encrypted with an on-processor key (something like intel SGX) -- reading OS memory, e.g dumping from linux or hardware, you can't read it unless you somehow extract the key (which are different on each chip) from the physical chip. Now, the process running using that encrypted memory generates TLS keys for you to send the data, and operates on it only inside this secure enclave. There is no way to access it without destroying the chip, and even in this scenario it will be extremely expensive and imo unlikely, certainly impossible at scale. Some scientists may be able to do it once in a lab.
    - sumeno 2y ago
      Ok, I'm imagining. Now, is any of that actually true?
      - sodality2 2y ago
        https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/
    - cyberpunk 2y ago
      BTW there is an entire industry popping up around exactly this sort of use case, it's called 'confidential computing' and CNCF have some software in the works (confidential containers iirc). I'm pretty excited to see what risc-v is going to bring to the party enclave wise.
  - Tagbert 2y ago
    It does need to process the data. The server has no persistent storage and no remote shell. It is a limited and locked down special-purpose iOS.

terramex 2y ago

Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hosted LLama 3 (or whatever will be released in next 6-12 months).

hmottestad 2y ago
Or Groq. They can do 1250 tokens/s with Llama 3 8B.

SoftTalker 2y ago

I see no real difference between 2 and 3. Once the data has left your device, it has left your device. There is no getting it back and you no longer have any control over it.

r00fus 2y ago
Hopefully they have some toggle in settings for this.
solarkraft 2y ago
Apple has demonstrated to be relatively trustworthy about privacy while most AI companies have demonstrated the opposite, so I do see a significant difference.
- SoftTalker 2y ago
  Google was considered very cool and trustworthy at one point also. "Do no evil" and all that.
  - babypuncher 2y ago
    Google's entire buisness model was built on hoovering up and selling access to user data in the form of AdSense. Without that data, their business falls apart. Apple's business model is to entice people into a walled garden ecosystem where they buy lots of expensive hardware sold on high margins. They don't need user data to make this work, which is why they can more comfortably push features like end-to-end and no-knowledge encryption.
  - doctor_eval 2y ago
    Google was cool, once upon a time, but they always used your personal info pretty openly. The CEO a himself famously said, “The Google policy on a lot of things is to get right up to the creepy line and not cross it.” Apple has taken a markedly different approach, and has done so for years - E2E encryption, hashing and segmenting routes on maps, Secure Enclave, etc. While I think it’s perfectly reasonable to “trust no one”, and I fully agree that there may be things we don’t know, I don’t think there it’s reasonable to put Apple on the same (exceedingly low) level as Google.

spike021 2y ago

It sounded like 3 is meant for non-personal stuff. Basically like a search engine style feature. When you want to look up things like say sports records and info, or a movie and info about it, etc.