2 ms·
Quantization and speculative decoding unlocked major savings for our smaller models. Still chasing that ideal cost/performance ratio.
by copperwire 3mo ago
Quantization and speculative decoding unlocked major savings for our smaller models. Still chasing that ideal cost/performance ratio.