3 ms·
That's very impressive. What's the best way to run these kernels natively on a Mac? I saw that there's a way to plug Claude into Apple's Foundation Models frame
by mike_hearn 4mo ago
That's very impressive. What's the best way to run these kernels natively on a Mac? I saw that there's a way to plug Claude into Apple's Foundation Models framework, and there's a CLI tool that can access models via that framework. It might be useful to have something so fast and good available via a small CLI tool for various purposes, especially when connected with a small suite of tools I have for things like file editing, showing, simple agentic purposes etc.
- thomspoon 4mo agoEither ollama or omlx, both are pretty dang performant. Omlx lets you run Claude code locally though as long as you bootstrap it with the right model
- mike_hearn 4mo agoOmlx is really nice, thanks for the recommendation!
- karussell 4mo agoWhy would you need Omlx? For speed up?
- pornel 4mo agoHas extra KV cache on SSD, and lots more options to tweak. There's experimental TurboQuant and multi token prediction support.