> But that fact isnt sufficient to make "semantic distinctions" -- because the proposition isn't true in virtue of that frequency. The trick here is that langu

> But that fact isnt sufficient to make "semantic distinctions" -- because the proposition isn't true in virtue of that frequency. The trick here is that language models are (currently!) demonstrating you /can/ get most of the way to semantic distinctions just by analyzing symbol-level statistics. Whether you can get all the way is an open question. I agree with you that "movie theaters" don't "sell candy" /because/ of some statistical artifact in large bodies of text. Movie theaters sell candy because people want to eat candy when they watch movies (and are willing to pay for it, etc.). But this wraps back around: the statistical artifacts happen to exist in large bodies of text because it is true. So, with enough text, and the right kind of analysis, you can tease the semantics back out. The power in language models is not that they "understand" text "the right way", from first principles, with a symbolic language model. The power is that they don't have to get most of the way there. Perhaps they'll get all the way there! And if they do, what then? Are we so sure that we don't do the same thing?