3 ms·
It could be that the rotation in the helix manifold whatever is a low level representation of the logical steps (carry the 2, add the next column,...) it's des
by markasoftware 2mo ago
It could be that the rotation in the helix manifold whatever is a low level representation of the logical steps (carry the 2, add the next column,...) it's describing. The point stands that the explanation it generates doesn't necessarily in all cases reflect what it "actually did" but your counterexample doesn't hold.
- dist-epoch 2mo agoThere have been multiple studies on how LLMs add numbers. They use the "Clock" algorithm, the "Pizza" algorithm, a few other ones. > All networks we study implement the same simple neuron model in their first-layer MLPs: degree-1 sinusoidal fits in layer 1, with deeper layers combining into degree-2 sinusoidal interactions. https://neurips.cc/virtual/2025/loc/san-diego/133808 https://neurips.cc/virtual/2025/loc/san-diego/133808 https://arxiv.org/abs/2502.00873 https://arxiv.org/abs/2502.00873 When you ask them they don't mention these at all, they give you high-school math: https://chatgpt.com/share/6a82afdd-872c-83eb-aad7-622d27f2dcdb https://chatgpt.com/share/6a82afdd-872c-83eb-aad7-622d27f2dc... > did you use the "cos" or "sin" function at all during this addition computation? > No. There is no need for trigonometric functions like sin or cos. The computation only uses basic arithmetic and place-value reasoning. Of course, if someone were implementing arithmetic in a computer, it is theoretically possible to express addition using extremely complicated formulas involving sin and cos. But in the reasoning I described, no trigonometric functions were involved at all. I simply decomposed the numbers into hundreds and smaller parts and added them.