2 ms·
If anyone wants to try this model on a Mac (it looks like they used something like DeepSeek-R1-Distill-Llama-8B) my new llm-mlx plugin can run it like this:
by simonw 2y ago
If anyone wants to try this model on a Mac (it looks like they used something like DeepSeek-R1-Distill-Llama-8B) my new llm-mlx plugin can run it like this:
brew install llm # or pipx install llm or uv tool install llm
llm install llm-mlx
llm mlx download-model mlx-community/DeepSeek-R1-Distill-Llama-8B
llm -m mlx-community/DeepSeek-R1-Distill-Llama-8B 'poem about an otter'
It's pretty performant - I got 22 tokens/second running that just now: https://gist.github.com/simonw/dada46d027602d6e46ba9e4f484775b5 https://gist.github.com/simonw/dada46d027602d6e46ba9e4f48477...