3 ms·

They got 1K tok/s with Deepseek v4 Pro. That's kinda cool..

• gaeld 5mo ago
  Thanks. To be fair, this number is what we expect to get once we port DeepSeek V4 in our engine on the upcoming generation of GPUs!
• gbnwl 5mo ago
  No they didn't, they predict they'll get that much. Also worth noting the prediction assumes running at MXFP4/FP8 quantization.
• dippatel1994 5mo ago
  Exactly! Any optimization for local inference is a welcome change IMHO!