x_flynn

3 ms·

1. 1.▲
   by x_flynn 1y ago

   What the model is doing in latent space is auxilliary to anthropomorphic interpretations of the tokens, though. And if the latent reasoning matches a ground-truth procedure (A*), then we'd expect it to be projectable to semantic tokens
2. 2.▲
   by x_flynn 1y ago

   It's more about the lack of semantic meaning in the intermediate tokens, not that they aren't effective (even when the intermediates are wrong)
3. 3.▲
   by x_flynn 1y ago

   I like to think of the intermediate tokens as low-dimensional hidden states. Also see the Coconut paper/citation
4. 4.▲
   by x_flynn 1y ago

   Exactly, the traces lack semantics and shouldn't be anthropomorphized. (I'm one of the students in the lab that wrote this, but not one of the authors)