Human play was used to bootstrap AlphaGo. But LLMs already have vast amount of data relating to human introspection. The problem is they cannot operationalize i

Human play was used to bootstrap AlphaGo. But LLMs already have vast amount of data relating to human introspection. The problem is they cannot operationalize it, as they have no analogs of our thinking/speaking distinction. Speaking and thinking is the same for them: producing a token is a prerequisite to produce a next token. Building a system that can leverage that knowledge to bootstrap "thinking circuits" can prove challenging, that's for sure, but I expect that collecting quality "thinking out loud" data is more challenging.