The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens.

The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens. Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking. Interpretability is a mixed bag even with trained tools on top of existing models.