> "The thing is that LLM's explanations of its "reasoning" are hallucinated. It has no access to its internal workings, and is no more insightful about those th

> "The thing is that LLM's explanations of its "reasoning" are hallucinated. It has no access to its internal workings, and is no more insightful about those than humans are (ie. not at all)." Yes this was my point when I said "If you ask a person what they were thinking, the person will just rationalize in the same way the bot will do it." We are saying the same thing. Or maybe you have more faith than me that a human's explanations of their own "reasoning" is not 'hallucinated' by post-hoc rationalization? > no access to its internal workings For the purposes of understanding the internal monologue of 'agentic' LLMs that run in some kind of loop, the human observer actually can get access to at least those parts of the internal workings of the AI to extract explanations like the ancestor commenter wanted. So in that sense, the AI is more reliably explainable than a human. It would be as if we had access to a human's internal monologue at all times (although the internal monologue of course is not the only part of cognition). That was my point in the 'red teaming' example.