4 ms·
> The model you're using to speculate could be anything, but if it's not guessing what the main model would predict, it's useless. So what I said is correct th
by Zetaphor 8mo ago
> The model you're using to speculate could be anything, but if it's not guessing what the main model would predict, it's useless.
So what I said is correct then lol. If you're saying I can use a model that isn't just a smaller quant of the larger model I'm trying to speculatively decode, except that model would never get an accurate prediction, then how is that in any way useful or desirable?