3 ms·

Does ChatGPT really blow away the Turing test? The criteria listed on Wikipedia is: > The evaluator would be aware that one of the two partners in conversatio

Does ChatGPT really blow away the Turing test? The criteria listed on Wikipedia is: > The evaluator would be aware that one of the two partners in conversation was a machine > If the evaluator could not reliably tell the machine from the human, the machine would be said to have passed the test. ChatGPT isn't there by a long shot. If you aren't really considering that an online agent may be a bot and are just casually interacting with it, then it might pass. But within 10 minutes, it is not going to become anywhere near as friendly as a person normally would in 10 minutes of conversation, especially if you know that one is a bot and are looking for differences. Turing Test is not casually passing as human in casual interaction. It is stumping a human dedicated to rooting out that it is a machine who is aware that one of the participants is a machine.

trealira 3y ago
> Turing Test is not casually passing as human in casual interaction. It is stumping a human dedicated to rooting out that it is a machine who is aware that one of the participants is a machine. It's a good point to bring up that what counts as "passing as human" is ambiguous, because people are using different definitions. I'm reminded of how trans people say that passing is a spectrum, and not a binary. There's a difference between passing in the distance, passing from 20 meters away, passing when interacting with service workers (like cashiers, receptionists, etc.) or in other quick interactions, and passing with your friends, family, and coworkers. Maybe someone trans passes in quick interactions, but not in an extended face-to-face conversation with a stranger. I've also heard that people relatively ignorant of trans people are less likely to perceive them as their birth sex compared to people who are liberal or who aren't but spend a lot of time reading about them anyway (e.g. certain feminists and conservatives). Those people are likely to see someone wear a dress and take the mental shortcut of categorizing them as a woman, not paying attention to details that others would have, unless it's extremely obvious that they don't pass. The same can be applied to AI. A language model might pass as a human to someone who doesn't know about AI; or to someone who knows about AI, but isn't expecting it in a certain context; but still not manage to stump the person who suspects and tries to figure out who's an AI and who's a human.
- fortysixed 3y ago
  That's an interesting comparison because in both cases the intent is to deceive, and no matter how successful the deception is, the deceiver is still not what they purport to be. Hopefully future law and policy regarding AI will be crafted with that in mind, unlike that other set of deceivers who have had their deception enacted into law as if it is reality, in many jurisdictions.

trealira 3y ago

> Turing Test is not casually passing as human in casual interaction. It is stumping a human dedicated to rooting out that it is a machine who is aware that one of the participants is a machine. It's a good point to bring up that what counts as "passing as human" is ambiguous, because people are using different definitions. I'm reminded of how trans people say that passing is a spectrum, and not a binary. There's a difference between passing in the distance, passing from 20 meters away, passing when interacting with service workers (like cashiers, receptionists, etc.) or in other quick interactions, and passing with your friends, family, and coworkers. Maybe someone trans passes in quick interactions, but not in an extended face-to-face conversation with a stranger. I've also heard that people relatively ignorant of trans people are less likely to perceive them as their birth sex compared to people who are liberal or who aren't but spend a lot of time reading about them anyway (e.g. certain feminists and conservatives). Those people are likely to see someone wear a dress and take the mental shortcut of categorizing them as a woman, not paying attention to details that others would have, unless it's extremely obvious that they don't pass. The same can be applied to AI. A language model might pass as a human to someone who doesn't know about AI; or to someone who knows about AI, but isn't expecting it in a certain context; but still not manage to stump the person who suspects and tries to figure out who's an AI and who's a human.

fortysixed 3y ago
That's an interesting comparison because in both cases the intent is to deceive, and no matter how successful the deception is, the deceiver is still not what they purport to be. Hopefully future law and policy regarding AI will be crafted with that in mind, unlike that other set of deceivers who have had their deception enacted into law as if it is reality, in many jurisdictions.