3 ms·
They probably use it on all models. Fast is probably just a resource pool with less congestion and therefore faster throughput per user but less efficent.
by gunalx 5mo ago
They probably use it on all models. Fast is probably just a resource pool with less congestion and therefore faster throughput per user but less efficent.
- deleted 5mo ago[deleted]
- cma 5mo agoIf it speeds prefill too I guess so.