3 ms·

Convinced that Apple has shot themselves in the foot / tied their hands behind their back / insert analogy here, re: privacy <> AI. Interesting to see how it p

Convinced that Apple has shot themselves in the foot / tied their hands behind their back / insert analogy here, re: privacy <> AI. Interesting to see how it plays out. Meta and Google have much more permissive privacy policies / stances, which means Meta + Google models are going to get much better faster. Apple does potentially have an edge with their Mx series of chips re: inference flops. I bet they're hoping that the model quality vs. model size curve continues dropping such that they can run sufficiently powerful LLMs on-device. We'll see.

cma 2y ago
Apple has a huge legacy burden of defaulting laptops to 2010 levels of memory capacity at 8GB all the way up to mid 2024. All the great AI inference stuff in the M-series is wasted due to that until you get ~80% penetration of the new 16GB min spec. If they get compelling local inference throughout the OS it may help them drive adoption of new models faster, but App devs won't have incentive to integrate local AI stuff until there is more market penetration of the higher RAM capacities. Apple could perhaps monetarily incentivise them to make up for it. But I think they are just going to heavily have to lean on cloud for everything.
- threeseed 2y ago
  Apple Intelligence is supported on any M1 Mac: https://www.apple.com/apple-intelligence https://www.apple.com/apple-intelligence And not sure there are many use cases for running mid-sized models locally. Smaller models fit in 8GB and work fine for action handling, tagging, classification etc. And for text/image generation you just get such a better user experience using a cloud hosted model.
  - cma 2y ago
    Fit in 8GB while running browsers and other apps that would want AI? Only tiny models that also work on phones.
threeseed 2y ago
a) Apple already reaches out to ChatGPT (and soon to be others) for the full LLM experience. And there isn’t much benefit to be had from building their own competitor. b) Privacy is essential if you want to an LLM to interact with you with private information. You can’t just upload unencrypted photos, messages, health data etc to the cloud where it will be accessible by any government with a search warrant. And so it does makes sense to keep it all private and encrypted on device or used in private but restricted compute. c) Hype is very much coming out of the AI space and so I don’t see the market punishing Apple for not doing more. If anything they should’ve done nothing rather than released the technically impressive but largely useless Apple Intelligence product.

cma 2y ago

Apple has a huge legacy burden of defaulting laptops to 2010 levels of memory capacity at 8GB all the way up to mid 2024. All the great AI inference stuff in the M-series is wasted due to that until you get ~80% penetration of the new 16GB min spec. If they get compelling local inference throughout the OS it may help them drive adoption of new models faster, but App devs won't have incentive to integrate local AI stuff until there is more market penetration of the higher RAM capacities. Apple could perhaps monetarily incentivise them to make up for it. But I think they are just going to heavily have to lean on cloud for everything.

threeseed 2y ago
Apple Intelligence is supported on any M1 Mac: https://www.apple.com/apple-intelligence https://www.apple.com/apple-intelligence And not sure there are many use cases for running mid-sized models locally. Smaller models fit in 8GB and work fine for action handling, tagging, classification etc. And for text/image generation you just get such a better user experience using a cloud hosted model.
- cma 2y ago
  Fit in 8GB while running browsers and other apps that would want AI? Only tiny models that also work on phones.

threeseed 2y ago

a) Apple already reaches out to ChatGPT (and soon to be others) for the full LLM experience. And there isn’t much benefit to be had from building their own competitor. b) Privacy is essential if you want to an LLM to interact with you with private information. You can’t just upload unencrypted photos, messages, health data etc to the cloud where it will be accessible by any government with a search warrant. And so it does makes sense to keep it all private and encrypted on device or used in private but restricted compute. c) Hype is very much coming out of the AI space and so I don’t see the market punishing Apple for not doing more. If anything they should’ve done nothing rather than released the technically impressive but largely useless Apple Intelligence product.