3 ms·
No, the words are meaningful to it. It's effectively using the CoT text as a "scratch space" for intermediate steps it can't calculate on one iteration through
by AlexCoventry 1y ago
No, the words are meaningful to it. It's effectively using the CoT text as a "scratch space" for intermediate steps it can't calculate on one iteration through the transformer. These papers give examples of how it works:
- https://physics.allen-zhu.com/part-2-grade-school-math/part-2-1 https://physics.allen-zhu.com/part-2-grade-school-math/part-...
- https://physics.allen-zhu.com/part-3-knowledge/part-3-3 https://physics.allen-zhu.com/part-3-knowledge/part-3-3
- modeless 1y agoI mean, this theory is directly contradicted by the paper under discussion. If you want to assert this then you need to be arguing why the paper is wrong.