LLM reasoning happens in a single forward pass. Produced tokens must conform to the training token distribution. It is necessary to grant the next forward pass

LLM reasoning happens in a single forward pass. Produced tokens must conform to the training token distribution. It is necessary to grant the next forward pass access to the outputs of the final layers in a way that is not subject to training loss. You could say that we are punishing LLMs for investing into tokens for the future. The LLM receives a greater reward for producing the right answer shape than the correct answer. When the limits of attention have been reached, the model fails to show better performance even with chain of thought prompting. Either the model needs to repeat layers recurrently or it needs an affordance model to understand the relative difficulty of a problem and split it into sub problems that it can actually solve.