4 ms·
Wow, this seems to be a new way to convert PyTorch models to a format that runs across CPU, GPU & Apple's Neural Engine (ANE). [0] Does this completely replace
by bensyverson 4mo ago
Wow, this seems to be a new way to convert PyTorch models to a format that runs across CPU, GPU & Apple's Neural Engine (ANE). [0]
Does this completely replace the previous API, CoreML? [1]
[0]: https://apple.github.io/coreai-optimization/
[1]: https://developer.apple.com/documentation/coreml/
- earthnail 4mo agoYes. From the CoreAI docs: "If your app uses model types other than neural networks, such as decision trees or tabular feature engineering, see Core ML."
- pzo 4mo agoseems they planning to replace it but overall now I'm really confused about this and mlx and coremltools. They should do better work explaining the benefits (and cons) of it and any feature parity between coreai, coreml and mlx.
- LoganDark 4mo agoMy reading of it is: - Core ML is for models designed only for Apple platforms - MLX is for models that don't need to be fast - Core AI is for models that run everywhere already and also need to be fast
- wahnfrieden 4mo agoI use CoreML for models designed for other platforms. I port the models to it but it works for that without much trouble. MLX is not for end user deployment.
- jkman 4mo agoThis view is a bit off. First, keep in mind that MLX was and will not be able to access the ANE, so it's a total non-starter for anything user-facing. Based on updates to coreml docs, they're trying to sell CoreML as the tool for tabular or domain-specific applications and CoreAI for NNs moving forward.
- LoganDark 4mo ago> keep in mind that MLX was and will not be able to access the ANE That's the rationale behind it not being fast. > so it's a total non-starter for anything user-facing Yep.
- ABS 4mo agolooks to me like the docs don't give a feature-parity table, but they do draw the "role" lines once you read across them: - Core ML narrows to classic, non-neural ML (its own docs now point you there for "decision trees or tabular feature engineering") - Core AI takes neural nets and transformers (the new .aimodel format, the new profiler) - MLX stays the separate bring-your-own-weights track (its WWDC sessions draw no line back to Core AI at all) coreai-opt is the successor to coremltools on the optimization side.
- wahnfrieden 4mo agoRequires OS 27+, so CoreML is still useful for backwards compatibility.
- sgt 4mo agomacOS users aren't that good at upgrading regularly, but iOS users are at least obsessive about upgrading to the latest OS. I guess the system almost forces us.
- jjice 4mo agoThe workaround my friend uses (unintentionally) is being completely out of storage on her phone.
- wahnfrieden 4mo agoI still deploy CoreML features to iOS 15. Many devices in use can’t upgrade to 26/27
- sgt 4mo agoOf course, you caught me thinking like an Apple engineer. I forgot about older devices in circulation, and that's super important.
- trollbridge 4mo agoThis is just a bit exciting, although I wonder how the performance of this will stack up next to the stuff we already do with, e.g., a metal-optimised model which we then load into llama-cpp or whatever. (unsloth is a good example of doing this for you "batteries included").
- ElFitz 4mo agoA few months back someone reverse-engineered private ANE APIs and shown some significant performance improvements compared to CoreML and Metal, on both inference and training. - https://maderix.substack.com/p/inside-the-m4-apple-neural-engine https://maderix.substack.com/p/inside-the-m4-apple-neural-en... - https://news.ycombinator.com/item?id=47257931 https://news.ycombinator.com/item?id=47257931