3 ms·
The results are from 2023, not 1966. The fact that ELIZA (indeed surprisingly) did better than GPT-3.5 at fooling people into thinking it's human is discussed i
by twiss 3y ago
The results are from 2023, not 1966. The fact that ELIZA (indeed surprisingly) did better than GPT-3.5 at fooling people into thinking it's human is discussed in Section 4.4, "The ELIZA effect".
If you look at the examples in Appendix C, it seems that it's because ELIZA didn't match participants' current expectations of AI's behavior, and thus they thought it must be human:
Verdict: Human | Confidence: 50
Reason: hard to believe anyone would
purposefully make an AI this bad
Verdict: Human | Confidence: 70
Reason: doesn't respond to
adversarial attacks
Verdict: Human | Confidence: 72
Reason: Super erratic
Interestingly, this means that if you want to fool humans today, it might be more important to make an AI that's different from the ones in common use, rather than strictly better.
So yes, I agree the Turing Test tells us as much about human expectations as about AIs, and the researchers also acknowledge this.