3 ms·
You can also try speculative decoding with the E2B model. Under some conditions it can result in a decent speed up
by Havoc 7mo ago
You can also try speculative decoding with the E2B model. Under some conditions it can result in a decent speed up