3 ms·
An intuitive explanation for why reasoning tokens help is to remember that LLMs are just mathmatical functions f() that take in an input sequence x and produces
by TGower 2mo ago
An intuitive explanation for why reasoning tokens help is to remember that LLMs are just mathmatical functions f() that take in an input sequence x and produces the next token f(x). Without reasoning tokens, you require the function f() to immediately take you from x to the start of an output sequence that is a correct answer. With reasoning tokens, this is much relaxed, allowing for many repeated applications of f() to gradually steer you from the input sequence to the start of the correct output sequence.
It seems intuitive that continuing a correct output sequence is easier than the "discontinuity" of jumping from the input prompt to the output sequence.
- deleted 2mo ago[deleted]
- schmuhblaster 2mo agoIndeed, and maybe that's all there is to it. Still, I'd hope we will eventually better understand what's exactly happening in the wake of many repeated applications of f().
- nodja 2mo agoThe way I think about it is that it's unreasonable for a compute graph with a static number of operations to be able to answer both y=a*10 and something like y=((((x+x)*(x+1))/((2*x)+2))+((x*(x+3))/(x+3))-((x*x)/(x+1))+((x*x)/(x+1))-((x*(x+3))/(x+3))) in a single forward pass. Tokens are essentially a unit of work and can also be used for intermediate steps, not just final results.
- js8 2mo agoNot only that! LLM can also learn how to repeatedly apply f() for certain arguments, and run the shortcut. Sometimes, the shortcut learned is not actually repeated application of f(), which breaks semantic soundness of LLM's reasoning chain. These shortcuts can greatly help reasoning, but they are not guaranteed to be sound. So yeah, in that case, LLMs are reasoning right (using shortcuts) for the wrong reasons (learning them from data instead by running actual repeated application and memoizing the resulting rule).
- woopsn 2mo agoCoT evidently helps but a bias towards both correctness and innovation has to come from somewhere. The article implies OpenAI's proofs may be supported by Lean but regardless, who knows how many people are trying to disprove the next conjecture each day throwing away nonanswers. Fundamentally these systems are more powerful with better training and sampling methods, or better prompting. Tokens matter but you can rewrite many prompts to get a much better, faster answer using fewer tokens vs allowing "thinking" to go on and on.
- inigyou 2mo agoObviously likely corollary: text diffusion models don't need reasoning tokens
- calf 2mo agoThe article is about existing research suggesting that this intuition is incorrect, though.