3 ms·
Can you point to an LLM passing the turing test where they didn't invalidate the test by limiting the time or the topics? I've seen claims of passing but it's
by socalgal2 1y ago
Can you point to an LLM passing the turing test where they didn't invalidate the test by limiting the time or the topics?
I've seen claims of passing but it's always things like "with only 3 questions" or "with only 3 minutes of interrogation" or "With only questions about topic X". Those aren't Turing Tests. As an example, if you limit the test to short things than anything will pass "Limit to 1 word one question". User types "Hello", LLM response "Hi". PASS! (not!)
- acdha 1y agoThis is the best one I’ve seen but it has the notable caveat that it’s a relatively short 5 minute chat session: https://arxiv.org/pdf/2405.08007 https://arxiv.org/pdf/2405.08007 I do think we’re going to see this shift as AI systems become more commonplace and people become more practiced at recognizing the distinction between polished text and understanding.
- thinmalk 1y agoThe Turing test isn't a good test in general, but writing a paper about an AI "passing" it when it only wrote 4 short messages in the whole conversation is almost farcical. Hard coded chatbots were "passing the Turing test" in the 90's with this standard.