3 ms·

I'm curious if the M4 Max will be good enough.

• simonw 2y ago
  That will run Qwen 2.5 Coder 32B just fine. I'm using a M2 Max with 64GB of RAM.
  • FloatArtifact 2y ago
    How many tokens per second?
    • simonw 2y ago
      With MLX I got: Prompt: 49 tokens, 95.691 tokens-per-sec Generation: 723 tokens, 10.016 tokens-per-sec