2 ms·
Yes, I mean a Mac Studio with MLX. An M3 Ultra with 256GB of RAM is $5599. That should just about be enough to fit MiniMax M2 at 8bit for MLX: https://huggingf
by simonw 1y ago
Yes, I mean a Mac Studio with MLX.
An M3 Ultra with 256GB of RAM is $5599. That should just about be enough to fit MiniMax M2 at 8bit for MLX: https://huggingface.co/mlx-community/MiniMax-M2-8bit https://huggingface.co/mlx-community/MiniMax-M2-8bit
Or maybe run a smaller quantized one to leave more memory for other apps!
Here are performance numbers for the 4bit MLX one: https://x.com/ivanfioravanti/status/1983590151910781298 https://x.com/ivanfioravanti/status/1983590151910781298 - 30+ tokens per second.
- oxcidized 1y agoThanks for the info! Definitely much better than I expected.
- zht 1y agoIt’s kinda misleading to omit the generally terrible prompt processing speed on Macs 30 tokens per second looks good until you have to wait minutes for the first token
- simonw 1y agoThe tweet I linked to includes that information in the chart.