“Aha” as a single token records the LLM discovered it made a mistake and needs to pivot. On the next forward pass: it rediscovers the mistake, its “aha” noting

“Aha” as a single token records the LLM discovered it made a mistake and needs to pivot. On the next forward pass: it rediscovers the mistake, its “aha” noting that, and then provides the first token of the new idea. That “aha” contains information: the previous conclusion was somehow insufficient.