Structure in the “world”? You mean the stream of “tokens” we ingest? This just comes down to giving transformers more modalities, not just text tokens. There

Structure in the “world”? You mean the stream of “tokens” we ingest? This just comes down to giving transformers more modalities, not just text tokens. There is nothing about “2” that conveys any “twoness”, this is true of all symbols. The token “the text ‘iphone’” and the token “visual/tactile/etc data of iphone observation” are highly correlated. That is what you learn. I don’t know if you call that stipulation, maybe, but an LLM correlates too in its training phase. I don’t see the fundamental difference, only a lot of optimizing and architectural improvements to be made. Edit: and when I say “a lot”, I mean astronomical amounts of it. Human minds are pretty well tuned to this job, it’ll take some effort to come close.