3 ms·
People are pretty cagey about what they use in production, but yes, speculative sampling can offer massive speedups in inference
by _diq5 3y ago
People are pretty cagey about what they use in production, but yes, speculative sampling can offer massive speedups in inference