3 ms·
That will run Qwen 2.5 Coder 32B just fine. I'm using a M2 Max with 64GB of RAM.
by simonw 2y ago
That will run Qwen 2.5 Coder 32B just fine. I'm using a M2 Max with 64GB of RAM.
- FloatArtifact 2y agoHow many tokens per second?
- simonw 2y agoWith MLX I got: Prompt: 49 tokens, 95.691 tokens-per-sec Generation: 723 tokens, 10.016 tokens-per-sec