LLMs are trained on vast bodies of human-curated text that was made by humans or for humans - often both. Then they're tuned further on human feedback. And then

LLMs are trained on vast bodies of human-curated text that was made by humans or for humans - often both. Then they're tuned further on human feedback. And then they're pointed at tasks humans find to be useful, evaluated by how good they are at those tasks, and trained to get better at them. You aren't anthropomorphizing LLMs enough. Are LLM reasoning traces always faithful? Lmao no. Are human inner monologues always faithful? Lmao no. Both of them reflect thoughts somewhat, sometimes. Even in humans, conscious thought is the top of a vast iceberg of subconscious data processing.