4 ms·
People just confidently stating stuff like "current LLMs basically pass the Turing test" makes me feel like I've secretly been given much worse versions of all
by Latty 1y ago
People just confidently stating stuff like "current LLMs basically pass the Turing test" makes me feel like I've secretly been given much worse versions of all the LLMs in some kind of study. It's so divorced from my experience of these tools, I genuinely don't really understand how my experience can be so far from yours, unless "basically" is doing a lot of heavy lifting here.
- 1una 1y agoWell, LLMs do pass the Turing Test, sort of. https://arxiv.org/abs/2503.23674 https://arxiv.org/abs/2503.23674
- AberrantJ 1y agoHere's three comments, two were written by a human and one written by a bot - can you tell which were human and which were a bot? Didn’t realize plexiglass existed in the 1930s! I'm certainly not a monetization expert. But don't most consumers recoil in horror at subscriptions? At least enough to offset the idea they can be used for everything? Not sure why this isn’t getting more attention - super helpful and way better than I expected!
- deleted 1y ago[deleted]
- chrismorgan 1y agoOn such short samples: all three have been written by humans—or at least comments materially identical have been. The third has also been written by many a bot for at least fifteen years.
- AberrantJ 1y agoIf you're willing to say that a fifteen year old bot was "writing" then I think having a discussion on if current "bots" pass the Turing test is sort of moot
- deleted 1y ago[deleted]
- falcor84 1y agoAs far as I understand, Turing himself did not specify a duration, but here's an example paper that ran a randomized study on (the old) GPT 4 with a 5 minute duration, and the AI passed with flying colors - https://arxiv.org/abs/2405.08007 https://arxiv.org/abs/2405.08007 From my experience, AI has significantly improved since, and I expect that ChatGPT o3 or Claude 4 Opus would pass a 30 minute test.
- JohnFen 1y ago> "current LLMs basically pass the Turing test" makes me feel like I've secretly been given much worse versions of all the LLMs in some kind of study. I think you may think passing the Turing test is more difficult and meaningful than it is. Computers have been able to pass the Turing test for longer than genAI has been around. Even Turing thought it wasn't a useful test in reality. He meant it as a thought experiment.
- HarHarVeryFunny 1y agoHaving LLMs capable of generating text based on human training data obviously raises the bar for a text-only evaluation of "are you human?", but LLM output is still fairly easy to spot, and knowing what LLMs are capable of (sometimes superhuman), and not capable of, should make it fairly easy for a knowledgeable "turing test administrator" to determine if they are dealing with an LLM or not. It would be a bit more difficult if you were dealing with an LLM agent tasked with faking a turing test as opposed to a naieve LLM just responding as usual, but even there the LLM will reveal itself by the things that it plain can't do.
- nerdix 1y agoIf you need a specialized skill set (deep knowledge of current LLM limitations) to distinguish between human and machine then I would say the machine passes the turing test.
- HarHarVeryFunny 1y agoOK, but that's just your own "fool some of the people some of the time" interpretation of what a Turing test should be, and by that measure ELIZA passed the Turing test too, which makes it rather meaningless. The intent (it was just a thought experiment) of a Turing test, was that if you can't tell it's not AGI, then it is AGI, which is semi-reasonable, as long as it's not the village idiot administering the test! It was never intended to be "if it can fool some people, some of the time, then it's AGI".
- 1y ago
- x187463 1y agoPer the wiki article for Turing Test: > In the test, a human evaluator judges a text transcript of a natural-language conversation between a human and a machine. The evaluator tries to identify the machine, and the machine passes if the evaluator cannot reliably tell them apart. The results would not depend on the machine's ability to answer questions correctly, only on how closely its answers resembled those of a human. Based on this, I would agree with the OP in many contexts. So, yeah, 'basically', is a load bearing word here but seems reasonably correct in the context of distinguishing human vs bot in any scalable and automated way.
- dylan604 1y agoOr it could be a bad test evaluator. Just because one person was fooled does not mean the next will be too.
- HarHarVeryFunny 1y agoJudging a conversation transcript is a lot different from being able to interact with an entity yourself. Obviously one could make an LLM look human by having a conversation with it that deliberately stayed within what it was capable of, but judging such a transcript isn't what most people imagine as a turing test.
- cm2012 1y agoI have seen data from an AI call center that shows 70% of users never suspected they spoke to an AI
- HarHarVeryFunny 1y agoIsn't the idea of a Turing test whether someone (meaningfully knowledgeable about such things) can determine if they are talking to a machine, not can the machine fool some of the people some of the time? ELIZA passed the latter bar back in the 1960's ... a pretty low bar.
- bluefirebrand 1y agoWhy would they? Humans running call centers have been running on less than GPT level scripts for ages
- armchairhacker 1y agoIt can't mimic a human over the long term. It can solve a short, easy-for-human CAPTCHA.