4 ms·
This is called speculative decoding
by zwaps 2y ago
This is called speculative decoding
- qeternity 2y agoNo, speculative decoding is when you use a smaller draft model to propose tokens and then use the larger target model to verify the proposals. It has got nothing to do with domain specialization.