2 ms·
Something I've never understood about the Turing test: are you allowed to ask things about family, about what they had for breakfast, about where they're locate
by crazygringo 3y ago
Something I've never understood about the Turing test: are you allowed to ask things about family, about what they had for breakfast, about where they're located and what's the weather out, about whether they saw the movie than opened last weekend?
Because on the one hand, there are so many obvious gotchas. But if we insist on a test that covers any and all topics, this means that the AI not only has to be intelligent and human-like, but that it can invent an entire totally fictional life full of internally consistent fictional relationships and events, and is also somehow reasonably up-to-date on real world news. Which is way more complicated than anything we're asking a human to do!
So I always figured it was meant to be restricted to some kind of highly controlled domain, like starting with an article to read and then discussing its themes in a "disembodied" way, referring only to generally-known history, and never giving anything away about yourself or what year it is or even who's president. Not using slang language, or having any real "personality" at all. But I've never heard of those kinds of restrictions explicitly being part of the test, so it's left me very confused. (And how do you draw the lines anyways?)
- zabzonk 3y agothe ai is allowed to lie or invent, just like the human.
- TillE 3y agoOne of the many problems with the Turing test is that stuff like this is not well-defined, and it's up to any researcher to fill in the details.
- pdonis 3y ago> Something I've never understood about the Turing test: are you allowed to ask things about family, about what they had for breakfast, about where they're located and what's the weather out, about whether they saw the movie than opened last weekend? Turing's original paper had the interrogator asking the subject to write a sonnet on the subject of the Forth bridge, and later on he describes a dialogue in which the writer of a sonnet is asked questions about it that bring in seasons, Mr. Pickwick, and Christmas. So it certainly seems like he did not intend for the questions to be limited to a restricted domain. > this means that the AI not only has to be intelligent and human-like, but that it can invent an entire totally fictional life full of internally consistent fictional relationships and events In Turing's original formulation, where the AI has to convince the interrogator that it's a human, yes, it would either have to invent a fictional human background or it would have to actually believe it was a human with such a background. However, since the real point of the test is to determine whether the AI can "think", not whether it is a human, one could imagine letting the AI give perfectly honest answers about its background--"I am an AI, I began running on April 16, 2092, at 9:02:37.436 Eastern Daylight Time, in the MIT AI Lab..."--as long as it was able to continue to respond to questions about it the same way a human would to questions about their background. (Think of HAL 9000 in 2001: A Space Odyssey, who is perfectly aware that he's an AI and knows plenty of facts about his origin and development.) > and is also somehow reasonably up-to-date on real world news Only as up to date as an average human would be expected to be. The AI could answer "I don't know, I'm not up to date on that" or words to that effect just as a human would to questions about events they weren't aware of.
- inimino 3y agoThe "imitation game" that was Turing's model was a parlor game with a man and a woman behind a screen. The interlocutor could ask either of them any question. The winner was whoever convinced the questioner that they were the woman. The game was all about deception and tricky questions and no holds barred. No, GPT-4 isn't remotely anywhere near being in the same ballpark as passing the Turing test.
- gregw2 3y agoYour simple clear explanation of this reminds me of a context where I have seen this sort of female imitation game used “in real life”, without any reference to Turing. In Minecraft, there are some girls-only servers. In order to join, you have to answer a series of (canned) multiple choice questions to get admitted. There were indeed some tricky questions. I was not able to get them right first time despite being male, a good test taker and married for a fair number of years. Not sure what the false positive rate was though. You can find videos of this stuff on YouTube.
- Veedrac 3y agoThe Turing Test is not intended to be fair. It is intended to be sufficient.
- usrbinbash 3y ago> are you allowed to ask things about family, about what they had for breakfast, about where they're located and what's the weather out, about whether they saw the movie than opened last weekend? The interrogees are even allowed to plead the 5th and refuse to answer any question, or to tell the interrogator to leave them alone. Aka. any response that a human is able to do, the machine is also allowed to produce. And that is the biggest flaw of the Turing test, and one of the reasons why ML research is generally not overly focused on it, or on passing it: It doesn't test the AIs ability to give intelligent responses, it tests the interrogator's ability to differentiate between artificially and naturally produced responses. https://en.wikipedia.org/wiki/Turing_test#Na%C3%AFvet%C3%A9_of_interrogators https://en.wikipedia.org/wiki/Turing_test#Na%C3%AFvet%C3%A9_...