4 ms·
Speculative decoding is lossless because the main model checks whether it agrees with what the drafter outputted
by StevenWaterman 2mo ago
Speculative decoding is lossless because the main model checks whether it agrees with what the drafter outputted