4 ms·

Would it be possible to train a language translation model to map from latent space -> English? If my understanding is correct, a plain-text token is just a po

Would it be possible to train a language translation model to map from latent space -> English? If my understanding is correct, a plain-text token is just a point in latent space mapped to an embedding vector. Any reasoning done in latent space is therefore human-readable as a sequence of raw tokens. I'm not sure what the token sequence would look like at this point -- I assume they're full or partial (mainly English) words, connected together by the abstract N-dimensional latent space concept of the token, not connected grammatically. Something like: > prompt: add 2 + 2 > reasoning: respond computation mathematics algebra scalar integer lhs 2 rhs 2 op summation > lhs 2 rhs 2 op summation gives 4 > computation ops remain none result 4 > response: 4 Something like that; probably even less sensical. Regardless, that could be "language translated" to English easily. I have not read the paper so this may have been addressed.

sailingparrot 1y ago
> Any reasoning done in latent space is therefore human-readable as a sequence of raw tokens. The mapping from latent space to logits (a score for how suitable the model think each possible token is to be the next one in the sentence) is one of the most costly operation of the entire LLM, so when doing reasoning in latent space you typically want to introduce some kind of recurrence somewhere before this mapping takes place, and only go through it once the model is "done" reasoning, at which point it becomes human interpretable. > would it be possible to train a language translation model to map from latent space -> English It's hard and expensive and slow but some progress is being made here: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...
vonneumannstan 1y ago
I think the larger issue is eventually the models might 'think' by passing massive matrices of floating point numbers around and there's basically no hope of decoding those.

sailingparrot 1y ago

> Any reasoning done in latent space is therefore human-readable as a sequence of raw tokens. The mapping from latent space to logits (a score for how suitable the model think each possible token is to be the next one in the sentence) is one of the most costly operation of the entire LLM, so when doing reasoning in latent space you typically want to introduce some kind of recurrence somewhere before this mapping takes place, and only go through it once the model is "done" reasoning, at which point it becomes human interpretable. > would it be possible to train a language translation model to map from latent space -> English It's hard and expensive and slow but some progress is being made here: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...

vonneumannstan 1y ago

I think the larger issue is eventually the models might 'think' by passing massive matrices of floating point numbers around and there's basically no hope of decoding those.