> Any reasoning done in latent space is therefore human-readable as a sequence of raw tokens. The mapping from latent space to logits (a score for how suitable

> Any reasoning done in latent space is therefore human-readable as a sequence of raw tokens. The mapping from latent space to logits (a score for how suitable the model think each possible token is to be the next one in the sentence) is one of the most costly operation of the entire LLM, so when doing reasoning in latent space you typically want to introduce some kind of recurrence somewhere before this mapping takes place, and only go through it once the model is "done" reasoning, at which point it becomes human interpretable. > would it be possible to train a language translation model to map from latent space -> English It's hard and expensive and slow but some progress is being made here: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...