3 ms·

I Tested the M5 iPad Pro's Neural-Accelerated AI, and the Hype Is Real

33 points by alwillis 11mo ago

bigyabai 11mo ago
Like prior iPad GPU upgrades, most customers will never notice until it reaches the Mac lineup. Another victim of the iOS/iPadOS capability bottleneck.
- PatMunch3551 11mo ago
  [dead]
deleted 11mo ago
[deleted]
joakleaf 11mo ago
Related and test on MacBook Pro M5 vs M4: https://machinelearning.apple.com/research/exploring-llms-mlx-m5 https://machinelearning.apple.com/research/exploring-llms-ml... "Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU"
- seanmcdirmid 11mo ago
  Note this is the M5, not even the M5 Pro and definitely not the M5 Max or M5 Ultra. If they are getting these improvements on low end M series, I’m sort of interested in what happens with the M5 max when it’s ready (I. Not holding out hope that the M5 ultra will be done anytime soon).
greatgib 11mo ago
Prompt ingested (time to first token) in "18 seconds" with the new chip... end of the joke.
- bigyabai 11mo ago
  At this rate, Apple will be primed to replace OpenAI for realtime use somewhere around 2045.
  - seanmcdirmid 11mo ago
    That is actually a reasonable prediction. By 2045, even tablets and high end phones will be able to locally run large enough models for real time chat.
- seec 11mo ago
  But don't forget, "the hype is real". Apple fanboys are something else. At this point it's not even clear that the LLMs will have much use outside of chatbots spitting questionable reality faking but it's pretty clear that the small local models are largely useless. Outside of the poor user experience the lack of large dataset make them an exercice out technical feasibility more and anything else. Apple feeling like they have to partner with Google to replace their own "Apple Intelligence" should tell you everything you need to know about local AI but I guess believers gotta believe.
  - alwillis 11mo ago
    > Apple feeling like they have to partner with Google to replace their own "Apple Intelligence" They’re not replacing Apple Intelligence; the partnership with Google is for the backend of Siri.
    - seec 11mo ago
      Yeah but this is what I mean. They are unable to produce a decent AI solution so they have to outsource. And if it was just about being limited by on device processing power, they could just switch to using their cloud infrastructure. To me that's just dumb. They are spending money for a competitor solution just to keep control and appearances. At this point they could just forget about AI altogether and/or open their OSs to let others integrate their solutions. But Apple is way too proud to admit being wrong and we will get a half-assed solution that will both cost them money and be inferior to the competition because of their own self-imposed limitations. They have been riding on their privacy/security narrative for a while, yet they work with governments and ads/tracking hasn't gone anywhere. And they'll have to open up the App Store golden goose at some point anyway. They should recognise when the strategy isn't working for the long run and switch it up... But they have tried to compete with Google for maps, it is still largely inferior in many ways and they seem to be fine with that. So I think it will end up in a similar situation.
      - alwillis 11mo ago
        You know Google pays Apple ~$20 billion a year to be the default search engine, right? The rumored deal with Google is $1 billion, so Apple is still up $19 billion. There's only three frontier model makers: OpenAI, Anthropic and Google. So everyone that's using a frontier model behind the scenes is using one of these three. It's likely whatever happens, it'll be running on Apple's Private Compute Cloud [1]—it won't be mined for ads, etc. What you don't understand is on-device AI isn't going away due to using Gemini for Siri; they're two separate products. Using Gemini is the fastest way to add "world knowledge" to Siri. [1]: https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/
- alwillis 11mo ago
  > Prompt ingested (time to first token) in "18 seconds" with the new chip... end of the joke 18 seconds on the M5; 4.4x times faster than the previous M4 running one of Qwen’s 8 billion parameter local models. That’s quite impressive for a tablet and faster than most laptops available today.

33 points by alwillis 11mo ago

Like prior iPad GPU upgrades, most customers will never notice until it reaches the Mac lineup. Another victim of the iOS/iPadOS capability bottleneck.

PatMunch3551 11mo ago
[dead]

deleted 11mo ago

[deleted]

joakleaf 11mo ago

Related and test on MacBook Pro M5 vs M4: https://machinelearning.apple.com/research/exploring-llms-mlx-m5 https://machinelearning.apple.com/research/exploring-llms-ml... "Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU"

seanmcdirmid 11mo ago
Note this is the M5, not even the M5 Pro and definitely not the M5 Max or M5 Ultra. If they are getting these improvements on low end M series, I’m sort of interested in what happens with the M5 max when it’s ready (I. Not holding out hope that the M5 ultra will be done anytime soon).

greatgib 11mo ago

Prompt ingested (time to first token) in "18 seconds" with the new chip... end of the joke.

bigyabai 11mo ago
At this rate, Apple will be primed to replace OpenAI for realtime use somewhere around 2045.
- seanmcdirmid 11mo ago
  That is actually a reasonable prediction. By 2045, even tablets and high end phones will be able to locally run large enough models for real time chat.
seec 11mo ago
But don't forget, "the hype is real". Apple fanboys are something else. At this point it's not even clear that the LLMs will have much use outside of chatbots spitting questionable reality faking but it's pretty clear that the small local models are largely useless. Outside of the poor user experience the lack of large dataset make them an exercice out technical feasibility more and anything else. Apple feeling like they have to partner with Google to replace their own "Apple Intelligence" should tell you everything you need to know about local AI but I guess believers gotta believe.
- alwillis 11mo ago
  > Apple feeling like they have to partner with Google to replace their own "Apple Intelligence" They’re not replacing Apple Intelligence; the partnership with Google is for the backend of Siri.
  - seec 11mo ago
    Yeah but this is what I mean. They are unable to produce a decent AI solution so they have to outsource. And if it was just about being limited by on device processing power, they could just switch to using their cloud infrastructure. To me that's just dumb. They are spending money for a competitor solution just to keep control and appearances. At this point they could just forget about AI altogether and/or open their OSs to let others integrate their solutions. But Apple is way too proud to admit being wrong and we will get a half-assed solution that will both cost them money and be inferior to the competition because of their own self-imposed limitations. They have been riding on their privacy/security narrative for a while, yet they work with governments and ads/tracking hasn't gone anywhere. And they'll have to open up the App Store golden goose at some point anyway. They should recognise when the strategy isn't working for the long run and switch it up... But they have tried to compete with Google for maps, it is still largely inferior in many ways and they seem to be fine with that. So I think it will end up in a similar situation.
    - alwillis 11mo ago
      You know Google pays Apple ~$20 billion a year to be the default search engine, right? The rumored deal with Google is $1 billion, so Apple is still up $19 billion. There's only three frontier model makers: OpenAI, Anthropic and Google. So everyone that's using a frontier model behind the scenes is using one of these three. It's likely whatever happens, it'll be running on Apple's Private Compute Cloud [1]—it won't be mined for ads, etc. What you don't understand is on-device AI isn't going away due to using Gemini for Siri; they're two separate products. Using Gemini is the fastest way to add "world knowledge" to Siri. [1]: https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/
alwillis 11mo ago
> Prompt ingested (time to first token) in "18 seconds" with the new chip... end of the joke 18 seconds on the M5; 4.4x times faster than the previous M4 running one of Qwen’s 8 billion parameter local models. That’s quite impressive for a tablet and faster than most laptops available today.