3 ms·

I'm skeptical. Like Microsoft, Apple has a lot of money to throw at super large models. But the question is whether they are committed enough to AI to do it, an

I'm skeptical. Like Microsoft, Apple has a lot of money to throw at super large models. But the question is whether they are committed enough to AI to do it, and whether they have the required ML talent. The latter was actually a problem for Microsoft, otherwise they wouldn't cooperate with OpenAI. Regarding commitment: This also seems iffy to me. Generative AI doesn't fit very well into Apple's hardware centric philosophy.

Otek 3y ago
> Generative AI doesn't fit very well into Apple's hardware centric philosophy. Well, I disagree with this take. Apple is known for playing the long game and planning for many years ahead. CPU power is still growing and Apple now has their own CPUs on every device. Sure you won’t be able to run something similar to GPT-4 in foreseeable future, but I predict we will see multiple small, feature-oriented LLMs that can easily be fit into a smartphone or at least an iPad with M(n) processor
whywhywhywhy 3y ago
> Generative AI doesn't fit very well into Apple's hardware centric philosophy They have a huge unique niche to exploit with local private models, probably the only company on the planet with the hardware in place to take advantage of this today end-to-end, but unfortunately I don't think generative AI fits with Apple's culture for anything I just think they'd be terrified of it saying something wrong so would probably clip its wings to the point that it's mundane and kinda useless.
- smoldesu 3y ago
  I don't think it's a very huge or exploitable niche for two reasons: - Android, Windows, Linux and MacOS can already run local and private models just fine. Getting something product-ready for iPhone is a game of catch-up, and probably a losing battle if Apple insists on making you use their AI assistant over competing options. - The software side needs more development. The current SOTA inferencing techniques for Apple Silicon in llama.cpp are cobbled together with Metal shaders and NEON instructions, neither of which are ideal or specific to Apple hardware. If Apple wants to differentiate their silicon from the 2016 Macbooks running LLaMA with AVX, then they have to develop CoreML's API further.
  - capableweb 3y ago
    Not saying you're right/wrong, but argument from the other side: Apple's hardware ships with hardware specifically for ML, and a lot of it (maybe even all of the recently released?) has shared memory between GPU/CPU, meaning you could run larger models. Windows/Linux is currently stuck with having to run inference with either RAM or VRAM, limiting the options quite a bit.
  - jroesch 3y ago
    Things are already possible on today's hardware, see https://github.com/mlc-ai/mlc-llm https://github.com/mlc-ai/mlc-llm which allows many models to be run on M1/M2 Macs, WASM, iOS and more. The main limiting factor will be small enough, high quality enough models that performance is high enough ultimately this is HW limited and they will need to improve the neural engine/map more computation on to it to make the mobile exp. possible.
hn_throwaway_99 3y ago
> But the question is whether they are committed enough to AI to do it, and whether they have the required ML talent. Given how business leaders throughout tech feel that AI is going to be transformative, I don't think commitment is really going to be a problem. Many leaders feel that "you either get good at AI or you don't exist in 10 years". In terms of attracting talent, there are 3 main things top AI folks look for: 1. Money (they are people after all) 2. The infrastructure (both hardware and people/organization-wise) to support large AI projects. 3. The willingness to release these AI projects to a large swath of people (to have "impact" as folks like to say). E.g. Google had 1 and 2 but their reticence to release their models and corporate infighting made many of the top Google researchers leave for gigs elsewhere. I think it remains to be seen how Apple will handle #3 as well.
- jroesch 3y ago
  To chime in Apple already has a lot of great ML talent they are just far more deliberate and slow to change their products. People forget that FaceID was/is one of the most cutting edge ML features ever developed/deployed when it was released a few years ago. Siri is sort of a red herring because its built by teams and tech that existed before Apple acquired most of its ML talent and some of its inability to evolve has been due to internal politics not the inability to build tech. iOS 17 is an example of Apple moving towards more deep learning speech/text work. I would bet heavily we will see them catch up with well integrated pieces as they have Money, infra, and already the ability to go wide (i.e all iOS users, again think FaceID).
matwood 3y ago
> Generative AI doesn't fit very well into Apple's hardware centric philosophy. I think it fits pretty well since Apple controls almost 100% of their stack. If they need hardware specific tweaks to make AI models run better, they can do that. On M* Macs for example, the unified memory model lets them do a number of AI tasks even with the lower powered GPU.
- cubefox 3y ago
  Good language models are far too demanding to run well on consumer products.

Otek 3y ago

> Generative AI doesn't fit very well into Apple's hardware centric philosophy. Well, I disagree with this take. Apple is known for playing the long game and planning for many years ahead. CPU power is still growing and Apple now has their own CPUs on every device. Sure you won’t be able to run something similar to GPT-4 in foreseeable future, but I predict we will see multiple small, feature-oriented LLMs that can easily be fit into a smartphone or at least an iPad with M(n) processor

whywhywhywhy 3y ago

> Generative AI doesn't fit very well into Apple's hardware centric philosophy They have a huge unique niche to exploit with local private models, probably the only company on the planet with the hardware in place to take advantage of this today end-to-end, but unfortunately I don't think generative AI fits with Apple's culture for anything I just think they'd be terrified of it saying something wrong so would probably clip its wings to the point that it's mundane and kinda useless.

smoldesu 3y ago
I don't think it's a very huge or exploitable niche for two reasons: - Android, Windows, Linux and MacOS can already run local and private models just fine. Getting something product-ready for iPhone is a game of catch-up, and probably a losing battle if Apple insists on making you use their AI assistant over competing options. - The software side needs more development. The current SOTA inferencing techniques for Apple Silicon in llama.cpp are cobbled together with Metal shaders and NEON instructions, neither of which are ideal or specific to Apple hardware. If Apple wants to differentiate their silicon from the 2016 Macbooks running LLaMA with AVX, then they have to develop CoreML's API further.
- capableweb 3y ago
  Not saying you're right/wrong, but argument from the other side: Apple's hardware ships with hardware specifically for ML, and a lot of it (maybe even all of the recently released?) has shared memory between GPU/CPU, meaning you could run larger models. Windows/Linux is currently stuck with having to run inference with either RAM or VRAM, limiting the options quite a bit.
- jroesch 3y ago
  Things are already possible on today's hardware, see https://github.com/mlc-ai/mlc-llm https://github.com/mlc-ai/mlc-llm which allows many models to be run on M1/M2 Macs, WASM, iOS and more. The main limiting factor will be small enough, high quality enough models that performance is high enough ultimately this is HW limited and they will need to improve the neural engine/map more computation on to it to make the mobile exp. possible.

hn_throwaway_99 3y ago

> But the question is whether they are committed enough to AI to do it, and whether they have the required ML talent. Given how business leaders throughout tech feel that AI is going to be transformative, I don't think commitment is really going to be a problem. Many leaders feel that "you either get good at AI or you don't exist in 10 years". In terms of attracting talent, there are 3 main things top AI folks look for: 1. Money (they are people after all) 2. The infrastructure (both hardware and people/organization-wise) to support large AI projects. 3. The willingness to release these AI projects to a large swath of people (to have "impact" as folks like to say). E.g. Google had 1 and 2 but their reticence to release their models and corporate infighting made many of the top Google researchers leave for gigs elsewhere. I think it remains to be seen how Apple will handle #3 as well.

jroesch 3y ago
To chime in Apple already has a lot of great ML talent they are just far more deliberate and slow to change their products. People forget that FaceID was/is one of the most cutting edge ML features ever developed/deployed when it was released a few years ago. Siri is sort of a red herring because its built by teams and tech that existed before Apple acquired most of its ML talent and some of its inability to evolve has been due to internal politics not the inability to build tech. iOS 17 is an example of Apple moving towards more deep learning speech/text work. I would bet heavily we will see them catch up with well integrated pieces as they have Money, infra, and already the ability to go wide (i.e all iOS users, again think FaceID).

matwood 3y ago

> Generative AI doesn't fit very well into Apple's hardware centric philosophy. I think it fits pretty well since Apple controls almost 100% of their stack. If they need hardware specific tweaks to make AI models run better, they can do that. On M* Macs for example, the unified memory model lets them do a number of AI tasks even with the lower powered GPU.

cubefox 3y ago
Good language models are far too demanding to run well on consumer products.