>An LLM cannot form secret chains of thought (though in theory a closed system outside the end-users' control could hide tokens from at least the user), nor can

>An LLM cannot form secret chains of thought (though in theory a closed system outside the end-users' control could hide tokens from at least the user), nor can it model decent metacognition. This is not an architectural limitation. All the Frontier Labs have pretty much noted you actually have to work to keep RL-tuned thinking models with human readable chains of thoughts. They quickly devolve into (still working) gibberish pretty quickly otherwise. And there are a few research variants out there that keep chains of thought entirely internal. It's not a thing for frontier models because frontier labs do not want secret chains of thought, not because it's an architectural impossibility. >I would be able to tell you why everything exists, especially if I merely added it in anticipation of something that will use it later. There's a ton of conscious planning that goes into these things. I’m not denying deliberation. I’m denying that introspection gives you transparent access to the mechanism of deliberation. Those are different claims. You may indeed have spent hours consciously turning the problem over. But when you later say “this is why I did X,” that report may still be a compressed narrative laid over a messier process involving habit, salience, constraint satisfaction, tacit pattern matching, and only partly articulated foresight. The point is not subconscious versus conscious. The point is that reportability is not the same thing as causal access. And the evidence that our reportability is faithful is much weaker than people usually assume.