5 ms·

We posted the same thing, in essence, at the same time. This piece is completely nonsensical in every way, and I presume it is targeted at laymen who'll just go

We posted the same thing, in essence, at the same time. This piece is completely nonsensical in every way, and I presume it is targeted at laymen who'll just go along with it. Like anyone who sees that last bit about MLX and CoreML and doesn't realize the author seems to not have a clue what they're talking about should understand they're being duped. Apple adopted a new cooling technique on their highest end device to differentiate and give spec sheet chasers something to be hyped about. It should help reduce throttling for the very odd event where someone is running a mobile device at 100% continuously (which is actually super rare in normal usage). It's already in the Pixel 9 Pro, for instance, and is a new "must have". It has nothing to do with whatever app these guys were building. The rest of the nonsense is just silly. If you are building an app for a mobile device and it pegs the CPU and GPU, you're going to have a bad time. That's the moment you realize it's time to go back to the drawing board.

MediaSquirrel 1y ago
Our app wasn't running on CPU or GPU –– the actual software we built was running entirely on Apple Neural Engine and it was crazy fast because we designed the architecture explicitly to run that specific chip. We were just calling the iPhone's built-in face tracking system via the Vision Framework to animate the avatars. That's the thing that was running on GPU.
- llm_nerd 1y ago
  Okay, though I'm not sure what that has to do with my comment. I understood that from the post: you were concurrently maxing out multiple parts of the SoC and it was overheating as they all contributed to the thermal load. This isn't new or novel -- benchmarks that saturate both the CPU and GPU are legendary for throttling -- though the claim that somehow normal thermal management didn't protect the hardware is novel, albeit entirely unsubstantiated. That is neither here nor there on CoreML -- which also uses the CPU, GPU, and ANE, and sometimes a combination of all of them -- or the weird thing about MLX.
  - MediaSquirrel 1y ago
    I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward. The only reason to use CoreML these days is to tap into the Neural Engine. When building for CoreML, if one layer of your model isn't compatible with the Neural Engine, it all falls back to the CPU. Ergot, CoreML is the only way to access the ANE, but it's a buggy all-or-nothing gambit. Have you ever actually shipped a CoreML model or tried to use the ANE?
    - llm_nerd 1y ago
      >Apple's focus is obviously on MLX / Metal going forward. This is nonsensical. MLX and CoreML are orthogonal. MLX is about training models. CoreML is about running models, or ML-related jobs. They solve very different problems, and MLX patches a massive hole that existed in the Apple space. Anyone saying MLX replaces CoreML, as the submission does, betrays that they are simply clueless. >The only reason to use CoreML these days is to tap into the Neural Engine. Every major AI framework on Apple hardware uses CoreML. What are you even talking about? CoreML, by the very purpose of its design, uses any of the available computation subsystems, which on the A19 will be the matmul units on the GPU. Anyone who thinks CoreML exists to use the ANE simply doesn't know what they're talking about. Indeed, the ANE is so limited in scope and purpose that it's remarkably hard to actually get it to use the ANE. >Have you ever actually shipped a CoreML model or tried to use the ANE? Literally a significant part of my professional life, which is precisely why this submission triggered every "does this guy know what he's talking about" button.
      - MediaSquirrel 1y ago
        Yes, MLX is for research, but MLX-Swift is for production and it works quite well for supported models! Unlike CoreML, the developer community is vibrant and growing. https://github.com/ml-explore/mlx-swift https://github.com/ml-explore/mlx-swift Maybe I am working on a different set of problems than you are. But why would you use CoreML if not to access ANE? There are so many other, better newer options like llama.cpp, MLX-Swift, etc. What are you seeing here that I am missing? What kind of models do you work with?

MediaSquirrel 1y ago

Our app wasn't running on CPU or GPU –– the actual software we built was running entirely on Apple Neural Engine and it was crazy fast because we designed the architecture explicitly to run that specific chip. We were just calling the iPhone's built-in face tracking system via the Vision Framework to animate the avatars. That's the thing that was running on GPU.

llm_nerd 1y ago
Okay, though I'm not sure what that has to do with my comment. I understood that from the post: you were concurrently maxing out multiple parts of the SoC and it was overheating as they all contributed to the thermal load. This isn't new or novel -- benchmarks that saturate both the CPU and GPU are legendary for throttling -- though the claim that somehow normal thermal management didn't protect the hardware is novel, albeit entirely unsubstantiated. That is neither here nor there on CoreML -- which also uses the CPU, GPU, and ANE, and sometimes a combination of all of them -- or the weird thing about MLX.
- MediaSquirrel 1y ago
  I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward. The only reason to use CoreML these days is to tap into the Neural Engine. When building for CoreML, if one layer of your model isn't compatible with the Neural Engine, it all falls back to the CPU. Ergot, CoreML is the only way to access the ANE, but it's a buggy all-or-nothing gambit. Have you ever actually shipped a CoreML model or tried to use the ANE?
  - llm_nerd 1y ago
    >Apple's focus is obviously on MLX / Metal going forward. This is nonsensical. MLX and CoreML are orthogonal. MLX is about training models. CoreML is about running models, or ML-related jobs. They solve very different problems, and MLX patches a massive hole that existed in the Apple space. Anyone saying MLX replaces CoreML, as the submission does, betrays that they are simply clueless. >The only reason to use CoreML these days is to tap into the Neural Engine. Every major AI framework on Apple hardware uses CoreML. What are you even talking about? CoreML, by the very purpose of its design, uses any of the available computation subsystems, which on the A19 will be the matmul units on the GPU. Anyone who thinks CoreML exists to use the ANE simply doesn't know what they're talking about. Indeed, the ANE is so limited in scope and purpose that it's remarkably hard to actually get it to use the ANE. >Have you ever actually shipped a CoreML model or tried to use the ANE? Literally a significant part of my professional life, which is precisely why this submission triggered every "does this guy know what he's talking about" button.
    - MediaSquirrel 1y ago
      Yes, MLX is for research, but MLX-Swift is for production and it works quite well for supported models! Unlike CoreML, the developer community is vibrant and growing. https://github.com/ml-explore/mlx-swift https://github.com/ml-explore/mlx-swift Maybe I am working on a different set of problems than you are. But why would you use CoreML if not to access ANE? There are so many other, better newer options like llama.cpp, MLX-Swift, etc. What are you seeing here that I am missing? What kind of models do you work with?