9 ms·
Apple Core AI Framework
- bensyverson 5mo agoWow, this seems to be a new way to convert PyTorch models to a format that runs across CPU, GPU & Apple's Neural Engine (ANE). [0] Does this completely replace the previous API, CoreML? [1] [0]: https://apple.github.io/coreai-optimization/ [1]: https://developer.apple.com/documentation/coreml/
- earthnail 5mo agoYes. From the CoreAI docs: "If your app uses model types other than neural networks, such as decision trees or tabular feature engineering, see Core ML."
- pzo 5mo agoseems they planning to replace it but overall now I'm really confused about this and mlx and coremltools. They should do better work explaining the benefits (and cons) of it and any feature parity between coreai, coreml and mlx.
- LoganDark 5mo agoMy reading of it is: - Core ML is for models designed only for Apple platforms - MLX is for models that don't need to be fast - Core AI is for models that run everywhere already and also need to be fast
- wahnfrieden 5mo agoI use CoreML for models designed for other platforms. I port the models to it but it works for that without much trouble. MLX is not for end user deployment.
- jkman 5mo agoThis view is a bit off. First, keep in mind that MLX was and will not be able to access the ANE, so it's a total non-starter for anything user-facing. Based on updates to coreml docs, they're trying to sell CoreML as the tool for tabular or domain-specific applications and CoreAI for NNs moving forward.
- LoganDark 5mo ago> keep in mind that MLX was and will not be able to access the ANE That's the rationale behind it not being fast. > so it's a total non-starter for anything user-facing Yep.
- ABS 5mo agolooks to me like the docs don't give a feature-parity table, but they do draw the "role" lines once you read across them: - Core ML narrows to classic, non-neural ML (its own docs now point you there for "decision trees or tabular feature engineering") - Core AI takes neural nets and transformers (the new .aimodel format, the new profiler) - MLX stays the separate bring-your-own-weights track (its WWDC sessions draw no line back to Core AI at all) coreai-opt is the successor to coremltools on the optimization side.
- wahnfrieden 5mo agoRequires OS 27+, so CoreML is still useful for backwards compatibility.
- sgt 5mo agomacOS users aren't that good at upgrading regularly, but iOS users are at least obsessive about upgrading to the latest OS. I guess the system almost forces us.
- jjice 5mo agoThe workaround my friend uses (unintentionally) is being completely out of storage on her phone.
- wahnfrieden 5mo agoI still deploy CoreML features to iOS 15. Many devices in use can’t upgrade to 26/27
- sgt 5mo agoOf course, you caught me thinking like an Apple engineer. I forgot about older devices in circulation, and that's super important.
- trollbridge 5mo agoThis is just a bit exciting, although I wonder how the performance of this will stack up next to the stuff we already do with, e.g., a metal-optimised model which we then load into llama-cpp or whatever. (unsloth is a good example of doing this for you "batteries included").
- ElFitz 5mo agoA few months back someone reverse-engineered private ANE APIs and shown some significant performance improvements compared to CoreML and Metal, on both inference and training. - https://maderix.substack.com/p/inside-the-m4-apple-neural-engine https://maderix.substack.com/p/inside-the-m4-apple-neural-en... - https://news.ycombinator.com/item?id=47257931 https://news.ycombinator.com/item?id=47257931
- MysticOracle 5mo agoWWDC 2026 Core AI videos Meet Core AI - https://developer.apple.com/videos/play/wwdc2026/324/ https://developer.apple.com/videos/play/wwdc2026/324/ Dive into Core AI model authoring and optimization - https://developer.apple.com/videos/play/wwdc2026/325/ https://developer.apple.com/videos/play/wwdc2026/325/ Integrate on-device AI models into your app using Core AI - https://developer.apple.com/videos/play/wwdc2026/326/ https://developer.apple.com/videos/play/wwdc2026/326/
- franze 5mo agoi am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundationmodels https://developer.apple.com/documentation/updates/foundation... (not much info yet) but i maintain https://github.com/Arthur-Ficial/apfel https://github.com/Arthur-Ficial/apfel so i might be biased
- crancher 5mo agoApfel is very useful, thanks for the effort.
- cat5e 5mo agoI second this, I’m more excited about dumb local models than something I could never run locally.
- trollbridge 5mo agoThanks for building this! Something I grab on a regular basis, especially for doing simple education of folks about the basics of using LLMs by showing something that's not just a chatbot.
- robgough 5mo agoHave you seen that they've added an `fm` tool? It was mentioned in the Platforms State of the Union. Here's what you get when you run it... https://gist.github.com/robgough/7893602895e75801174750761988ffca https://gist.github.com/robgough/7893602895e7580117475076198...
- an0malous 5mo agoThis is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it
- sealeck 5mo agoHave we reached the limits of scaling? Sadly it appears that larger model still equals better model
- stogot 5mo agoIt’s still diminishing returns yes? It isn’t Moore’s Law
- pixelready 5mo agoI think there’s still an open question around are the ultra-large next-gen models worth it? For those of us without early access to Mythos, it’s hard to verify whether it’s been held back from the public due to actually being “too dangerously powerful to release yet” as implied or because the gains aren’t outpacing the costs.
- mikestorrent 5mo agoWell, let's not forget that text models are not the only models! Video models are much slower and need comparatively more resources, and all they can do even at that size is generate videos a few seconds long. Clearly a ton more work is going to go into those, and demand for them will probably increase as more creative tools get authored using them as a central part of the workflow. Low-res local rendering for preview might be a thing, but the lion's share of the work for high-res, near-realtime rendering is going to be done on huge clusters for a long time yet.
- niek_pas 5mo agoThis is definitely a good point. I imagine the max capacity for video models is significantly lower than for text models (there just aren't as many professionals in video as there are people who write text or code) but I could be wrong.
- criddell 5mo agoIs there something like this on Linux? For example, if I’m an application developer can I assume GNU Core AI (or whatever it is or would be called) will be there if the kernel is >= some particular version?
- wtallis 5mo agoOn non-Apple platforms, you generally have at least 2+(number of supported silicon vendors) different AI frameworks to worry about. I guess Apple's there now too, between Core ML, MLX, Core AI. I haven't seen any sign that the framework fragmentation problem is going away anytime soon. NVIDIA wants everyone to do all training and inference with CUDA and to deny that NPUs have any usefulness. Everybody making an NPU has a different framework tailored to their architecture and the limitations they inherited from hardware designed before LLMs existed, and most of them have a another framework for targeting a GPU. And the OS vendor has one or two frameworks they would prefer you use rather than something hardware-specific.
- nl 5mo agoFor practical purposes llama.cpp is this. You can link to it or use the network API.
- halJordan 5mo agoNo there isn't. RedHat and IBM do though, for their distros
- teravor 5mo agoonnxruntime, llama.cpp (more specifically, ggml), iree.dev is also trying
- dvt 5mo agoAI future is clearly local, and my recent pitch has been "infinite tokens." Because that's what my M1 MBP can do; and that's what my RTX3090 can do. I don't need to pay hundreds of dollars a month and no one else does either.
- deleted 5mo ago[deleted]
- ip26 5mo agoInfinite tokens rate-limited to 10 tok/s is 26MTok per month.
- doctorpangloss 5mo ago10? think closer to 5. 13M is like ~7 codex sessions…
- fedeb95 5mo agothe real money is in the coding surrounding models to make them efficient at specialized tasks. Casual users want general purpose models, and AI chat apps will stay for them. Most programs can benefit from a specialized AI that can be local, and #programs >> #users.
- AdamN 5mo agoAlso context - there's alot of context out there and it's faster to get it from servers. It doesn't matter how good the model is if it doesn't have context from data sources.
- pmontra 5mo agoIn the 80s we thought that the future of computing was clearly local, home computers, PCs, Macs, the office server (Novell, then Windows NT with disk shares) etc. Add 40 years and we are back to a centralized infrastructure with the modern equivalent of smart terminals. The AI future will be clearly... what it will be. Probably bouncing back and forth from local to centralized. However, if there are money to be made by selling things that people run locally, it seems that centralizing creates more power and hence more money.
- connectsnk 5mo agoDo we know what is the underlying model? Is it a custome model developed by Apple or one of gemma/deepseeks under the hood
- jacobr1 5mo agoThe new siri models will be some variant of the gemini models. This framework seems to be more generalized than that though.
- ankit219 5mo agothey are also working on activations (w4a8, w4a16 from what i know). if they deliver (and a big if), it means that given their market reach, they can dictate the way sub 100b parameter models are trained and served to a large extent, given their major usecase would be on device (macos and not ios for most of them).
- ABS 5mo ago[flagged]
- scosman 5mo agoFree server-size model access for apps with <2M downloads, getting the same privacy guarantees. Hopefully they scale this up to all apps in time (I assume hardware/cost constrained, but larger devs would pay). https://developer.apple.com/private-cloud-compute/ https://developer.apple.com/private-cloud-compute/
- wahnfrieden 5mo agoMy guess based on the Apple Intelligence Extensions mentions is that they will not scale that up anytime soon, but they will allow developers to integrate with other providers that the user has an account with.
- JV00 5mo agoDoes it mean I can run whatever I want on ANE? Last time I tried it seemed it could only be used by first party features such as Face ID
- wahnfrieden 5mo agoBeen doing that for years with CoreML
- jkman 5mo agoYou've already been able to do that once you convert your model to CoreML, it's only MLX that's never been able to use the ANE.
- ABS 5mo agosomething I haven't seen highlighted anywhere yet, while I find it very interesting, is the distributed inference across Macs (JACCL over Thunderbolt 5), an OpenAI-compatible mlx_lm.server, agentic-on-Mac. Apple keeps MLX (bring-your-own-weights) separate from Foundation Models / Core AI.
- meyusufdemirci 4mo agoWaiting for the on device models specially. They will be game changers if they will be clever models enough. BTW, I also have an recap article of Core AI framework, https://appcircle.io/blog/wwdc26-apple-core-ai-framework-explained https://appcircle.io/blog/wwdc26-apple-core-ai-framework-exp...