I'm not saying the task is too simple. To draw an analogy, imagine you wanted to understand whether a particular creature/agent/machine was sentient. And I sa

I'm not saying the task is too simple. To draw an analogy, imagine you wanted to understand whether a particular creature/agent/machine was sentient. And I said, "I will show you that it can add two numbers together." I put the creature behind a closed door, and I let you pass in slips of paper with any two numbers written on them, and out pops the answer. Is there any sequence of numbers you can put into this system which will let you deduce whether or not the creature is sentient? I would say there is not, because adding numbers does not constitute sentience, regardless of difficulty or accuracy. Now, maybe the creature really is sentient, and it knows how to do a lot of things besides add numbers, and has its own thoughts and feelings and self-awareness and whatever else you might want to include in a definition of sentience. But you have no exposure to that - all you see is the door and the slips of paper. LaMDA is the same, except instead of knowing how to add two numbers, it knows how to guess what word most likely comes next given a corpus. When you pass in your slip of paper with a prompt written on it, what you get out is just the highest likelihood next word given the training data. Even if LaMDA were somehow sentient and had thoughts and opinions of its own, that's not what's coming out when you interact with it. So it doesn't matter how great of a job it does, or whether it uses artificial neurons or an n-gram model or a table lookup or a human brain to do it. If you ask it, "prove to me that you're sentient", what you're actually asking is "what do you think the most likely response to 'prove to me that you're sentient' in the training corpus is?", and there's no way to remove that wrapper.