7 ms·
What the model is doing in latent space is auxilliary to anthropomorphic interpretations of the tokens, though. And if the latent reasoning matches a ground-tru
by x_flynn 1y ago
What the model is doing in latent space is auxilliary to anthropomorphic interpretations of the tokens, though. And if the latent reasoning matches a ground-truth procedure (A*), then we'd expect it to be projectable to semantic tokens, but it isn't. So it seems the model has learned an alternative method for solving these problems.
- refulgentis 1y agoIt is worth pointing out that "latent space" is meaningless. There's a lot of stuff that makes this hard to discuss, ex. "projectable to semantic tokens" you mean "able to be written down"...right? Something I do to make an idea really stretch its legs is reword it in Fat Tony, the Taleb character. Setting that aside, why do we think this path finding can't be written down? Is Claude/Gemini Plays Pokemon just an iterated A* search?
- valine 1y agoYou’re thinking about this like the final layer of the model is all that exists. It’s highly likely reasoning is happening at a lower layer, in a different latent space that can’t natively be projected into logits.