4 ms·
Apparently Google is capable of doing cool things. DeepMind’s speculative sampling achieves 2–2.5x decoding speedups in LLM. That brings cost down significantly
by visarga 4y ago
Apparently Google is capable of doing cool things. DeepMind’s speculative sampling achieves 2–2.5x decoding speedups in LLM. That brings cost down significantly, without degradation in quality.