4 ms·

"Just returning the next token" is an overly reductive description that misses what is powerful about these models. To accurately predict the next token require

"Just returning the next token" is an overly reductive description that misses what is powerful about these models. To accurately predict the next token requires the discovery of a model that has a correspondence with the system being modeled to varying degrees of fidelity. In the limit of infinite samples, the model must perfectly capture the structure and information dynamics of the system. Any deficiency in the model will necessarily reveal itself in a difference in output at some point. Of course LLMs aren't trained in the limit of infinite samples, but the question is how much of the system (a human mind producing language) has it modeled? If sentience or understanding is a necessary feature of a mind producing the kinds of texts we produce, it is an open question whether LLMs have or will recover the right sort of structure for these properties.

seanhunter 4y ago
You're trying to pull yourself up by your own bootstraps here. If sentience is required to produce the kinds of texts that LLMs produce then sentience is a feature of LLMs of course. But if sentience is required for my computer to produce the "Tada!" noise when it switches on then my computer is sentient also. (It's probably not sentient.) Nothing you have said establishes in any sense that LLMs are sentient, and the Chinese Room experiment shows that we can't just rely on our intuition.
- hackinthebochs 4y ago
  I'm not sure what to make of comments like this. The discussion around LLMs and their capacities has just been universally horrendous. >But if sentience is required for my computer to produce the "Tada!" noise when it switches on then my computer is sentient also. But there is a large and obvious difference between your PCs boots up sound process and the output of LLMs. For one, its far more plausible that sentience is required to produce the kinds of texts that LLMs produce than to produce a boot up sound. >Nothing you have said establishes in any sense that LLMs are sentient I wasn't trying to, I was simply pointing out the error in concluding a lack of sentience from the arguments given. If you want to have a discussion about how/why LLMs might be sentient, I'm happy to have it. But most people don't actually want to have that discussion.

seanhunter 4y ago

You're trying to pull yourself up by your own bootstraps here. If sentience is required to produce the kinds of texts that LLMs produce then sentience is a feature of LLMs of course. But if sentience is required for my computer to produce the "Tada!" noise when it switches on then my computer is sentient also. (It's probably not sentient.) Nothing you have said establishes in any sense that LLMs are sentient, and the Chinese Room experiment shows that we can't just rely on our intuition.

hackinthebochs 4y ago
I'm not sure what to make of comments like this. The discussion around LLMs and their capacities has just been universally horrendous. >But if sentience is required for my computer to produce the "Tada!" noise when it switches on then my computer is sentient also. But there is a large and obvious difference between your PCs boots up sound process and the output of LLMs. For one, its far more plausible that sentience is required to produce the kinds of texts that LLMs produce than to produce a boot up sound. >Nothing you have said establishes in any sense that LLMs are sentient I wasn't trying to, I was simply pointing out the error in concluding a lack of sentience from the arguments given. If you want to have a discussion about how/why LLMs might be sentient, I'm happy to have it. But most people don't actually want to have that discussion.