5 ms·
Apple Releases Open Source AI Models That Run On-Device
- gnabgib 2y agoDiscussion: [0] (33 points, 18 hours ago, 7 comments) [0]: https://news.ycombinator.com/item?id=40140675 https://news.ycombinator.com/item?id=40140675
- sp332 2y agoWhy is the 3B model worse than the 450M model on MMLU and TruthfulQA?
- simonw 2y agoHas anyone seen a working, clearly explained recipe for running this using the Python MLX library on macOS yet?
- ein0p 2y agoWhat’s there to explain? There’s a readme in the repo that shows how to do it.
- simonw 2y agoI tried and failed to follow that. I'm looking for a report from someone who has got it (or something like it) to work.
- ein0p 2y agoI’ve tried their 1.1B model. The only hiccup was that it seems to require mlx 0.10.0 which is what’s in requirements.txt. You also have to place the llama tokenizer file into the model dir - they do not distribute it. The models published for MLX do not seem to be instruction tuned, so with their default prompt they get repetitive. But I suppose you could convert the instruction tuned checkpoints with the script in the repo.
- mritchie712 2y agoguessing it'll be available thru ollama soon https://huggingface.co/apple/OpenELM/discussions/5 https://huggingface.co/apple/OpenELM/discussions/5
- Bloating 2y agoNow we can give credit to Apple for invented AI!
- Turing_Machine 2y agoApple rarely invents the first X. However, they often invent the first X that doesn't suck. Turns out that the second one is actually more important.
- thebruce87m 2y agoI never see apple claiming to invent something. I frequently see comments like yours however.
- deleted 2y ago[deleted]
- JKCalhoun 2y agoAn LLM in my pocket is truly a mind-blowing concept, I have to say. More than anything else — phone, camera, internet. The feels like a really big deal. And with regard to LLMs (AI?) in general, I don't think right now we have any idea what we will all be using them for in ten years. But it just feels like a fundamental change is coming from all this.
- ChrisArchitect 2y ago[dupe] Some more discussion: https://news.ycombinator.com/item?id=40140675 https://news.ycombinator.com/item?id=40140675
- solarkraft 2y agoI'm not knowledgeable enough to parse much out of the Readme. How "good" are the models approximately? What hardware do I need to run them? How fast are they?