Exactly! Nothing is like we would have expected. The seeming “understanding” and coherency of LLMs is a surprise, and it doesn’t seem like anyone knows exactl

Exactly! Nothing is like we would have expected. The seeming “understanding” and coherency of LLMs is a surprise, and it doesn’t seem like anyone knows exactly what it means about human language and cognition. It’s possible that by analyzing what LLMs are actually doing we will discover a “code” underlying language that is sort of like a universal grammar, or at least gain some insight into how they stitch together and combine remembered snippets of language so sensically. While what ChatGPT does seems more like “open-ended talking” than thinking, reasoning, and responding to its external and internal situation (which does not really exist, in the absence of plugins—there is just the talking), that talking can be used to (for example) plan a series of actions.