3 ms·
How does the non-COT system generate the second word of its not using the output as input? Or do you mean that non-COT systems use only the latest output word w
by lupire 3y ago
How does the non-COT system generate the second word of its not using the output as input? Or do you mean that non-COT systems use only the latest output word when computing the next word, not all the earlier words from the output?
- HarHarVeryFunny 3y agoEvery output word is always appended to the prompt, so if prompt is P then input after W1 (word 1) is output is P W1, then P W1 W2, etc. So, the LLM is only looking at it's own PAST words, not planning ahead. If you do COT (think step-by-step) then the difference is that it has broken the prompt request/problem down into steps, so while it's still only seeing it's own past words, those now include all of step-1, which helps it generate step-2, etc and eventually combine all steps it generated into a complete answer.