4 ms·

I'd recommend reading up on Private Cloud Compute, the system that Apple designed to implement this: https://security.apple.com/blog/private-cloud-compute/ http

I'd recommend reading up on Private Cloud Compute, the system that Apple designed to implement this: https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/ It's pretty impressive. I read it back in June when this came out, but the basic gist of it is that everything that _can_ be done on device _is_ done on device, and everything else is run in a _provably_ secure and private cloud environment.

amelius 2y ago
> Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises. How is this possible if the software runs on Apple hardware? Do the security researchers get access to the VLSI designs?
- judofyr 2y ago
  From the article (emphasis mine): > Private Cloud Compute hardware security starts at manufacturing, where we inventory and perform high-resolution imaging of the components of the PCC node before each server is sealed and its tamper switch is activated. When they arrive in the data center, we perform extensive revalidation before the servers are allowed to be provisioned for PCC. The process involves multiple Apple teams that cross-check data from independent sources, and the process is further monitored by a third-party observer not affiliated with Apple. At the end, a certificate is issued for keys rooted in the Secure Enclave UID for each PCC node. The user’s device will not send data to any PCC nodes if it cannot validate their certificates.
  - amelius 2y ago
    > high-resolution imaging of the components of the PCC node Does that mean they image the internals of the ICs? Or do they just make some pictures of the PCBs?
  - trogdor 2y ago
    > monitored by a third-party observer not affiliated with Apple Your friendly neighborhood NSA agent :p
ethbr1 2y ago
Honestly, this was the biggest thing that pushed me from Android to iOS. I don't trust Google to be (a) incentivized (because ad revenue) or (b) organizationally-capable (because product fiefdoms) to ship privacy arch to that level in Android. And if I'm going to adopt LLMs on mobile as part of my workflow, I want very strong guarantees about where that data is ending up.
- lrem 2y ago
  I believe that’s called Gemini Nano.
  - ethbr1 2y ago
    One model does not a strategy make. And I'm loathe to bet on a horse that makes money every time it bites me.
AlexErrant 2y ago
Not to dunk on you, but your misuse of "provably" makes me discount what you said. If we could "prove" security, we would. Proving security in a networked environment? Hahaha - there have been successful attacks on airgapped envs.
- MarkSweep 2y ago
  I think it’s “provably secure” in the same sense that end to end chat apps can be provably secure. You can compare codes in the apps to prove that your conversation are noting man-in-the-middles. You still have to trust the software running on both ends to be doing the right thing. Apple goes further than standard end-to-end encryption messaging by adding a software attestation component to the hand shake. And they say they will publish the server side software for researchers to poke at. And there is a certificate security style log so you can be sure that the server side software is published. I’m not saying the system is good or works, I’m just saying don’t totally discount the idea of designing a system that has provable security properties.
  - AlexErrant 2y ago
    I'm engaging in pedantry here, but computer science has proofs. When we have hardware level security vulnerabilities, I trust my device as far as I can throw it. Proofs are only sound and valid in maths. https://securityintelligence.com/news/apple-m-series-chips-hardware-flaw/ https://securityintelligence.com/news/apple-m-series-chips-h... Everything Apple's done yields verifiable security - but it's not "provably secure". The two are distinct, and when you try to sell to me with bad language I get squinty-eyed. Especially since "confidential computing" already exists on x86/AWS, and I struggle to see the difference. It just sounds like Apple marketing to me. > don’t totally discount the idea of designing a system that has provable security properties They're only as provable as your assumptions/givens. Given a hardware vuln, where is your security now?
    - shusson 2y ago
      Agreed, I think a lot of people do not appreciate the complexity of software systems. If we could prove software, we wouldn't have bugs. https://wiki.c2.com/?ProofsCantProveTheAbsenceOfBugs https://wiki.c2.com/?ProofsCantProveTheAbsenceOfBugs

amelius 2y ago

> Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises. How is this possible if the software runs on Apple hardware? Do the security researchers get access to the VLSI designs?

judofyr 2y ago
From the article (emphasis mine): > Private Cloud Compute hardware security starts at manufacturing, where we inventory and perform high-resolution imaging of the components of the PCC node before each server is sealed and its tamper switch is activated. When they arrive in the data center, we perform extensive revalidation before the servers are allowed to be provisioned for PCC. The process involves multiple Apple teams that cross-check data from independent sources, and the process is further monitored by a third-party observer not affiliated with Apple. At the end, a certificate is issued for keys rooted in the Secure Enclave UID for each PCC node. The user’s device will not send data to any PCC nodes if it cannot validate their certificates.
- amelius 2y ago
  > high-resolution imaging of the components of the PCC node Does that mean they image the internals of the ICs? Or do they just make some pictures of the PCBs?
- trogdor 2y ago
  > monitored by a third-party observer not affiliated with Apple Your friendly neighborhood NSA agent :p

ethbr1 2y ago

Honestly, this was the biggest thing that pushed me from Android to iOS. I don't trust Google to be (a) incentivized (because ad revenue) or (b) organizationally-capable (because product fiefdoms) to ship privacy arch to that level in Android. And if I'm going to adopt LLMs on mobile as part of my workflow, I want very strong guarantees about where that data is ending up.

lrem 2y ago
I believe that’s called Gemini Nano.
- ethbr1 2y ago
  One model does not a strategy make. And I'm loathe to bet on a horse that makes money every time it bites me.

AlexErrant 2y ago

Not to dunk on you, but your misuse of "provably" makes me discount what you said. If we could "prove" security, we would. Proving security in a networked environment? Hahaha - there have been successful attacks on airgapped envs.

MarkSweep 2y ago
I think it’s “provably secure” in the same sense that end to end chat apps can be provably secure. You can compare codes in the apps to prove that your conversation are noting man-in-the-middles. You still have to trust the software running on both ends to be doing the right thing. Apple goes further than standard end-to-end encryption messaging by adding a software attestation component to the hand shake. And they say they will publish the server side software for researchers to poke at. And there is a certificate security style log so you can be sure that the server side software is published. I’m not saying the system is good or works, I’m just saying don’t totally discount the idea of designing a system that has provable security properties.
- AlexErrant 2y ago
  I'm engaging in pedantry here, but computer science has proofs. When we have hardware level security vulnerabilities, I trust my device as far as I can throw it. Proofs are only sound and valid in maths. https://securityintelligence.com/news/apple-m-series-chips-hardware-flaw/ https://securityintelligence.com/news/apple-m-series-chips-h... Everything Apple's done yields verifiable security - but it's not "provably secure". The two are distinct, and when you try to sell to me with bad language I get squinty-eyed. Especially since "confidential computing" already exists on x86/AWS, and I struggle to see the difference. It just sounds like Apple marketing to me. > don’t totally discount the idea of designing a system that has provable security properties They're only as provable as your assumptions/givens. Given a hardware vuln, where is your security now?
  - shusson 2y ago
    Agreed, I think a lot of people do not appreciate the complexity of software systems. If we could prove software, we wouldn't have bugs. https://wiki.c2.com/?ProofsCantProveTheAbsenceOfBugs https://wiki.c2.com/?ProofsCantProveTheAbsenceOfBugs