3 ms·

I would pay extra to -not- have AI in my phone. AI belongs on mainframes, out of my way when I don't want it.

• xp84 2y ago
  This is an underrated take. Apple, especially, wants to leverage their excellent silicon to do “AI” locally in the name of privacy purely as a marketing advantage, but the tradeoffs there are major (burning my battery instead of data center AC power, plus, my phone will never be better or faster than some big metal GPU with fans), and the privacy issue could just as well be solved by doing a good job designing the server side architecture and educating people that unlike when talking to a person, it’s trivial for a model to “not learn anything from” a conversation, and that that’s in fact automatically how it works unless you’re logging the conversations and purposely feeding them back in.
  • xvector 2y ago
    You can use their cloud instead of local compute if you'd like: https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/ Building a privacy aware cloud is a lot harder than you think it is. Apple is going the whole way with remote attestation, hardware root of trust, software BOM, etc. https://security.apple.com/blog/pcc-security-research/ https://security.apple.com/blog/pcc-security-research/
    • xp84 2y ago
      They are, and I actually believe that they’ve done a good job of all that stuff (though suspect that using their weird Mac hardware in the DC can’t possibly be a truly superior option to the NVIDIA hardware the entire rest of the world is using, but let’s assume I’m wrong). But are you suggesting I will be able to flip a switch to use their cloud for all the same AI stuff instead of my battery-powered phone? That would be really welcome news! I haven’t used any of the betas so I know nothing about how “Apple Intelligence” works.
      • xvector 2y ago
        > But are you suggesting I will be able to flip a switch to use their cloud for all the same AI stuff instead of my battery-powered phone? Yup! That's the idea. Compute negligible tasks run locally, but for requests that would hog battery and compute (or even be impossible to run on your iPhone), the data center will be used! > though suspect that using their weird Mac hardware in the DC can’t possibly be a truly superior option to the NVIDIA hardware So they are doing this for two reasons: 1. They get remote attestation and hardware root of trust when using their own hardware. No need to load proprietary blobs from Nvidia, introduce a new unvetted third-party supply chain, etc 2. Apple Silicon is actually much more cost effective than Nvidia's GPUs for inference. You can get a 48GB Apple Silicon Mac for like $4k and run large models - it's $40k for similar VRAM from Nvidia. I imagine Apple's costs are much lower than $4k as well.
  • JadeNB 2y ago
    The problem is that remote solutions to the privacy problem inevitably involve my trusting someone else's privacy assurances. On-device processing reduces the trust required, although of course it doesn't eliminate it. The common response to this is to point out that I'm trusting Apple or Google anyway just by using their phones, which is true, but (1) since phones are directly accessible to security researchers, there's more potential to find out about malfeasance if data is exfiltrated from the phone in a way that it shouldn't be, and (2) even if I have complete trust in Apple or Google, I also have to trust that they'll be able to protect my data from malicious actors, and, the less data I give them, the less I have to worry when they fail.
    • xp84 2y ago
      Thanks for a well-reasoned take on why someone would prefer this side of that tradeoff. Fair points.