4 ms·
that sounds testable - if you skip the reasoning tokens, do you get the same result? if not, then there's certainly some bearing, but not necessarily in how we
by 8note 2mo ago
that sounds testable - if you skip the reasoning tokens, do you get the same result?
if not, then there's certainly some bearing, but not necessarily in how we read the tokens as text
- rcxdude 2mo agoIt is and has been - skipping the tokens causes performance to drop. But replacing the tokens with filler causes the performance to drop, but by a lot less. You can also train models to emit broken or unrelated thinking tokens and their performance is also not much worse than the ones that are trained to output somewhat coherent thinking traces. This points to a hypothesis that the content of the tokens is only slightly related to the mechanism by which it improves performance, and that primarily the extra tokens allow the original prompt to be processed more deeply by the model, because earlier tokens will essentially pass through the model many more times than later ones.
- neuroticnews25 2mo agoI'll never understand some downvoters here.
- ForHackernews 2mo agoAI boosters denying that their machine god is burbling sweet nothings to itself?