3 ms·
Yeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that spee
by 0xbadcafebee 1mo ago
Yeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that speed. The public service exists so they get some users on OpenRouter, and that shows them as #1 on speed, which proves their tech is very fast, which gets them billions in hardware sales/licensing. If you have big enough pockets they can probably dedicate capacity to you. But for reliably fast small models you might want to rent some GPUs.
- LoganDark 1mo agoGPUs can't reach these speeds. You could build a supercomputing cluster and still not reach these speeds.
- gpugreg 1mo agoMiMo-V2.5-Pro-UltraSpeed gets pretty close with over 1000 TPS on 8x B200. It has 1.02T total parameters and 42B active, compared to 27B total/active for Qwen3.8-27B. Also, B300 are out now. I think 1500 TPS for Qwen3.8-27B should be doable.
- LoganDark 1mo agoThat model uses a lot of tricks to achieve 1000 t/s. I would not use raw parameter counts alone for such comparisons, in general.
- nostrebored 1mo agoCeleris reaches ~50% of the speed on commodity hardware. celeris-magnus-1 is based on qwen3.8-27b. Maybe we will get there without custom silicon!