3 ms·
> If sentience is a necessary feature of a mind producing the kinds of texts we produce, it is an open question whether LLMs have or will recover the structure
by DanielVZ 4y ago
> If sentience is a necessary feature of a mind producing the kinds of texts we produce, it is an open question whether LLMs have or will recover the structure that endows sentience.
I kind of agree, but this is a big If. If it was necessary, then I’d expect some kind of breakthrough moment in which LLMs or some other model suddenly perfectly mimics consciousness and language instead of the gradual improvement based on the already massive training set that results in uncanny resemblance with infinite malleability.
- hackinthebochs 4y agoYeah, it would likely be a step change in quality at that point. I doubt the current architecture and training regime would take it in that direction. GPT essentially is a collection of personas to ideally mimic the widely different contexts in its training set. A singular sentient identity is counter-productive to modeling these varying contexts. Now if they could synthesize a large corpus of interactive chats and train on that, that may push it in the direction of a singular identity. Even then, I strongly suspect that some kind of recursive/recurrent architecture is necessary for sentience. Purely feedforward computation doesn't seem like it has the resources to result in the kind of tight unity of experience needed for sentience.