5 ms·

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083 https://github.com/ml-explore/mlx/pull/3083

by zcbenz 8mo ago

embedding-shape 8mo ago
Blog post dated 28 Jan 2026, the bug fix posted 29 Jan 2026, so I guess this story had a happy ending :) Still, sad state of affairs that it seems like Apple is still fixing bugs based on what blog posts gets the most attention on the internet, but I guess once they started that approach, it's hard to stop and go back to figuring out priorities on their own.
- jckahn 8mo ago
  Just goes to show that attention is all you need.
  - tensility 8mo ago
    A statement which goes to show that confusing correlation with causation is all you need.
- syntaxing 8mo ago
  I don’t think so. You can see the issue ticket linked in the PR. Whether that issue ticket is related to the blog post is unknown https://github.com/ml-explore/mlx-swift-examples/issues/462 https://github.com/ml-explore/mlx-swift-examples/issues/462
- llm_nerd 8mo ago
  MLX is a fairly esoteric library seeing very little usage, mostly to try to foment a broader NN space on Apple devices. This isn't something that is widely affecting people, and most people simply aren't trying to run general LLMs on their iPhone. I don't think that fix is specific to this, but it's absolutely true that MLX is trying to lever every advantage it can find on specific hardware, so it's possible it made a bad choice on a particular device.
- dahcryn 8mo ago
  I think you overestimate the power of a blogpost and the speed of bugfixing at Apple for something like this. I almost guarantee there is no way they can read this blogpost, escalate it internally, get the appropriate approval to the work item, actually work on the fix, get it through QA and get it live in production in 3 days. That would only happen on really critical issues, and this is definitely not critical enough for that.
  - embedding-shape 8mo ago
    Or, one of the developers of the library saw it, decided to fix it in their spare time (does that exist at Apple?) before it became a bigger thing. If not, talk about coincident that someone reported an issue and all of that you mentioned was already done before that happened, and the only thing missing was merging the code to the repository which was done after the issue was reported. Not unheard of, but feels less unlikely than "Engineer decided to fix it".
  - spacedcowboy 8mo ago
    Three days is, agreed, too short. A week is just about possible, though... I've seen a blog-post, authored a bug in Radar, assigned it to myself, and fixed it the same day. Whether it goes out in the next release is more a decision for the bug-review-board, but since the engineering manager (that would have been me) sits on that too, it's just a matter of timing and seeing if I can argue the case. To be fair, the closer we are to a release, the less likely a change is to be accepted unless you can really sweet-talk the rest of the BRB, and there's usually a week of baking before the actual release goes out, but that has sometimes been shrunk for developer-preview releases...
    - tensility 8mo ago
      The fixing of a bug at Apple is the easy and quick part. It's the submission process from then until it gets released as part of an OS update that is the ridiculously long (and too often difficult) part.
      - 8mo ago
- mrtesthah 8mo ago
  How do you know that it wasn’t merely that the blog post elicited multiple people to file the same duplicate bug in Apple’s radar system, which is how they ostensibly prioritize fixes?
  - embedding-shape 8mo ago
    I don't, but the effect is the same, "something might land in the news, lets fix it before it does, since multiple people reporting the same issue based on this public post someone made".
- rafaelcosta 8mo ago
  Extremely bad timing on my end then, should've waited for a few more days
zozbot234 8mo ago
So the underlying issue is that the iPhone 16 Pro SKU was misdetected as having Neural Accelerator (nax) support and this caused silently wrong results. Not a problem with the actual hardware.
- llm_nerd 8mo ago
  Apple's documentation is utter garbage, but this code almost seems like a separate issue (and notably the MLX library uses loads of undocumented properties in metal which isn't cool). It looks like the change used to allow the NAX kernel to be used on the iPhone 17 or upcoming 18 if you're on 26.2 or later, to instead only allow it on the iPhone 17 Pro or upcoming 18. I'm fairly sure the GPU arch on the A19 is 17. They changed it so it will only use that kernel on the 17 Pro or upcoming 18, which is notable as the A19 Pro in the 17 Pro has a significantly changed GPU, including GPU tensor cores. The only real change here is that it would limit to the pro variants for the "17" model.
  - zozbot234 8mo ago
    > The neural accelerator exists in iPhones going back many years. What has existed before is the Apple Neural Engine (ANE) which is very different from the newer Neural Accelerator support within the GPU blocks. In fact MLX does not even support ANE yet since at least in previous versions it was hardware-limited to computing FP16 and INT8 MADDs, and not even that fast.
    - llm_nerd 8mo ago
      Sure, I directly and explicitly talked about Apple's version of tensor cores in the GPU. But the ANE is by every definition a neural accelerator. Yes, I'm aware of Apple's weird branding for their tensor cores. "In fact MLX does not even support ANE yet" I didn't say otherwise. The ANE is a fantastic unit for small, power-efficient models, like extracting text from images, doing depth modelling, etc. It's not made for LLMs, or the other sorts of experimental stuff MLX is intended for. Though note that MLX's author's reason for not supporting the ANE is that it has a "closed-source" API (https://github.com/ml-explore/mlx/issues/18#issuecomment-1846492294 https://github.com/ml-explore/mlx/issues/18#issuecomment-184...), making it unsuitable for an open-source project, and given that MLX didn't want to just lean on CoreML. But anyways, the ANE is fantastically fast at what it does, while sipping juice. In any case, the code change shown should have zero impact on the running of MLX on an iPhone 16 Pro. MLX tries to really leverage platform optimizations so maybe another bifucation is making the wrong choice.
syntaxing 8mo ago
Kinda sucks how it seems like there’s no CI that runs on hardware.
liuliu 8mo ago
Why MLX doesn't just detect apple10 support (for Metal)? That excludes all the devices without NA.

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083 https://github.com/ml-explore/mlx/pull/3083

by zcbenz 8mo ago

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083 https://github.com/ml-explore/mlx/pull/3083

embedding-shape 8mo ago

Blog post dated 28 Jan 2026, the bug fix posted 29 Jan 2026, so I guess this story had a happy ending :) Still, sad state of affairs that it seems like Apple is still fixing bugs based on what blog posts gets the most attention on the internet, but I guess once they started that approach, it's hard to stop and go back to figuring out priorities on their own.

jckahn 8mo ago
Just goes to show that attention is all you need.
- tensility 8mo ago
  A statement which goes to show that confusing correlation with causation is all you need.
syntaxing 8mo ago
I don’t think so. You can see the issue ticket linked in the PR. Whether that issue ticket is related to the blog post is unknown https://github.com/ml-explore/mlx-swift-examples/issues/462 https://github.com/ml-explore/mlx-swift-examples/issues/462
llm_nerd 8mo ago
MLX is a fairly esoteric library seeing very little usage, mostly to try to foment a broader NN space on Apple devices. This isn't something that is widely affecting people, and most people simply aren't trying to run general LLMs on their iPhone. I don't think that fix is specific to this, but it's absolutely true that MLX is trying to lever every advantage it can find on specific hardware, so it's possible it made a bad choice on a particular device.
dahcryn 8mo ago
I think you overestimate the power of a blogpost and the speed of bugfixing at Apple for something like this. I almost guarantee there is no way they can read this blogpost, escalate it internally, get the appropriate approval to the work item, actually work on the fix, get it through QA and get it live in production in 3 days. That would only happen on really critical issues, and this is definitely not critical enough for that.
- embedding-shape 8mo ago
  Or, one of the developers of the library saw it, decided to fix it in their spare time (does that exist at Apple?) before it became a bigger thing. If not, talk about coincident that someone reported an issue and all of that you mentioned was already done before that happened, and the only thing missing was merging the code to the repository which was done after the issue was reported. Not unheard of, but feels less unlikely than "Engineer decided to fix it".
- spacedcowboy 8mo ago
  Three days is, agreed, too short. A week is just about possible, though... I've seen a blog-post, authored a bug in Radar, assigned it to myself, and fixed it the same day. Whether it goes out in the next release is more a decision for the bug-review-board, but since the engineering manager (that would have been me) sits on that too, it's just a matter of timing and seeing if I can argue the case. To be fair, the closer we are to a release, the less likely a change is to be accepted unless you can really sweet-talk the rest of the BRB, and there's usually a week of baking before the actual release goes out, but that has sometimes been shrunk for developer-preview releases...
  - tensility 8mo ago
    The fixing of a bug at Apple is the easy and quick part. It's the submission process from then until it gets released as part of an OS update that is the ridiculously long (and too often difficult) part.
    - 8mo ago
mrtesthah 8mo ago
How do you know that it wasn’t merely that the blog post elicited multiple people to file the same duplicate bug in Apple’s radar system, which is how they ostensibly prioritize fixes?
- embedding-shape 8mo ago
  I don't, but the effect is the same, "something might land in the news, lets fix it before it does, since multiple people reporting the same issue based on this public post someone made".
rafaelcosta 8mo ago
Extremely bad timing on my end then, should've waited for a few more days

zozbot234 8mo ago

So the underlying issue is that the iPhone 16 Pro SKU was misdetected as having Neural Accelerator (nax) support and this caused silently wrong results. Not a problem with the actual hardware.

llm_nerd 8mo ago
Apple's documentation is utter garbage, but this code almost seems like a separate issue (and notably the MLX library uses loads of undocumented properties in metal which isn't cool). It looks like the change used to allow the NAX kernel to be used on the iPhone 17 or upcoming 18 if you're on 26.2 or later, to instead only allow it on the iPhone 17 Pro or upcoming 18. I'm fairly sure the GPU arch on the A19 is 17. They changed it so it will only use that kernel on the 17 Pro or upcoming 18, which is notable as the A19 Pro in the 17 Pro has a significantly changed GPU, including GPU tensor cores. The only real change here is that it would limit to the pro variants for the "17" model.
- zozbot234 8mo ago
  > The neural accelerator exists in iPhones going back many years. What has existed before is the Apple Neural Engine (ANE) which is very different from the newer Neural Accelerator support within the GPU blocks. In fact MLX does not even support ANE yet since at least in previous versions it was hardware-limited to computing FP16 and INT8 MADDs, and not even that fast.
  - llm_nerd 8mo ago
    Sure, I directly and explicitly talked about Apple's version of tensor cores in the GPU. But the ANE is by every definition a neural accelerator. Yes, I'm aware of Apple's weird branding for their tensor cores. "In fact MLX does not even support ANE yet" I didn't say otherwise. The ANE is a fantastic unit for small, power-efficient models, like extracting text from images, doing depth modelling, etc. It's not made for LLMs, or the other sorts of experimental stuff MLX is intended for. Though note that MLX's author's reason for not supporting the ANE is that it has a "closed-source" API (https://github.com/ml-explore/mlx/issues/18#issuecomment-1846492294 https://github.com/ml-explore/mlx/issues/18#issuecomment-184...), making it unsuitable for an open-source project, and given that MLX didn't want to just lean on CoreML. But anyways, the ANE is fantastically fast at what it does, while sipping juice. In any case, the code change shown should have zero impact on the running of MLX on an iPhone 16 Pro. MLX tries to really leverage platform optimizations so maybe another bifucation is making the wrong choice.

syntaxing 8mo ago

Kinda sucks how it seems like there’s no CI that runs on hardware.

liuliu 8mo ago

Why MLX doesn't just detect apple10 support (for Metal)? That excludes all the devices without NA.