2 ms·

I've been hearing forever that latent reasoning is the next step. But none of the frontier labs seem to incorporate it. Any idea why? It doesn't sound so techni

by ngruhn 2mo ago

I've been hearing forever that latent reasoning is the next step. But none of the frontier labs seem to incorporate it. Any idea why? It doesn't sound so technically challenging. Sure you loose some interpretability but surely everyone would sacrifice that for a step-up in quality. Or at least offer a latent-reasoning-version of the model.

diwank 2mo ago
you lose CoT monitorability which is a big issue since models have become quite powerful and also often deceptive but i do think that efficiency pressure will keep nudging us toward latent reasoning. looped language models are an active research area. but imagine not being able to monitor mythos' thoughts as it's working through a national security need...
- kgeist 2mo ago
  LLMs already think in latent space. The generated reasoning tokens are only the surface of what's happening internally. An LLM may write one thing in the traces but decide differently in the latent space. The whole token-based "reasoning" thing was just a clever hack to extend the existing architecture without completely redoing it. In one of Anthropic's recent papers, they added an additional subnetwork trained to map internal states to readable text, so that's probably the vector of further development.
  - nmitchko 2mo ago
    The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens. Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking. Interpretability is a mixed bag even with trained tools on top of existing models.
nmitchko 2mo ago
I am on my own GPUs so thought it be cool to reduce token output. you can charge per latent loop as a token i guess. I wanted a knob to keep thinking going until it was sure it was done.

2 ms·

I've been hearing forever that latent reasoning is the next step. But none of the frontier labs seem to incorporate it. Any idea why? It doesn't sound so techni

by ngruhn 2mo ago

diwank 2mo ago
you lose CoT monitorability which is a big issue since models have become quite powerful and also often deceptive but i do think that efficiency pressure will keep nudging us toward latent reasoning. looped language models are an active research area. but imagine not being able to monitor mythos' thoughts as it's working through a national security need...
- kgeist 2mo ago
  LLMs already think in latent space. The generated reasoning tokens are only the surface of what's happening internally. An LLM may write one thing in the traces but decide differently in the latent space. The whole token-based "reasoning" thing was just a clever hack to extend the existing architecture without completely redoing it. In one of Anthropic's recent papers, they added an additional subnetwork trained to map internal states to readable text, so that's probably the vector of further development.
  - nmitchko 2mo ago
    The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens. Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking. Interpretability is a mixed bag even with trained tools on top of existing models.
nmitchko 2mo ago
I am on my own GPUs so thought it be cool to reduce token output. you can charge per latent loop as a token i guess. I wanted a knob to keep thinking going until it was sure it was done.