4 ms·
1960s chatbot ELIZA beat OpenAI's GPT-3.5 in a recent Turing test study
- FrustratedMonky 3y agoWonder if this would hold for GPT 4, and if GPT training on the tuning/human feedback was for more 'conversational' responses. Since recently, it does seem like GPT 4 is a little more 'dry' and guarded in its responses. A few months ago it would give me some sassy answers, now it is more business like.
- 7thaccount 3y ago"Surprisingly, ELIZA, developed in the mid-1960s by computer scientist Joseph Weizenbaum at MIT, scored relatively well during the study, achieving a success rate of 27 percent. GPT-3.5, depending on the prompt, scored a 14 percent success rate, below ELIZA. GPT-4 achieved a success rate of 41 percent, second only to actual humans" GPT-4 did better. I think the article stated 3.5 was like that by design.
- FrustratedMonky 3y agoYep missed this " And like GPT-3.5, GPT-4 has also been conditioned not to present itself as human"
- ngcc_hk 3y agoActually most of the 4 is better. Wonder why 1 is not.
- poulpy123 3y agoI really really doubt that an AI from the 60s is at the same level than gpt 3.5
- Teever 3y agoI don't doubt it. ELIZA isn't programmed to say "I'm sorry Dave, I'm afraid I can't do that" While ChatGPT is loaded to the gills with cautionary programming like that.
- apricot 3y agoYeah, the title is misleading here. What happened is that the Turing test judges, who had to decide whether they were talking to a human or an AI, judged ELIZA's output as much too poor for current AI, and therefore probably generated by a human intentionally being uncooperative.
- guidedlight 3y agoThis is almost exactly what the article says. > ...some interrogators reported thinking that ELIZA was “too bad” to be a current AI model, and therefore was more likely to be a human intentionally being uncooperative.
- poulpy123 3y agoThat's really funny
- deleted 3y ago[deleted]
- 3cats-in-a-coat 3y agoA good example of how misleading a test result can be when the test tests the wrong things.
- deleted 3y ago[deleted]