4 ms·

No it's not. The traditional interpretation of the Turing test requires interactivity. That is, the evaluator is allowed to ask questions and will receive a res

No it's not. The traditional interpretation of the Turing test requires interactivity. That is, the evaluator is allowed to ask questions and will receive a response from both a person and a machine. The idea is that there should be no sequence of questions you can ask that would reliably identify the machine. That's not even close to true for these "AI" systems.

akoboldfrying 1y ago
You're right about interactivity, something that I overlooked -- but I think it's nevertheless the case that a large fraction of human interrogators could not distinguish a human from a suitably-system-prompted ChatGPT even over the course of an interactive discussion. ChatGPT 4.5 was judged to be the human 73% of the time in this RCT study, where human interrogators had 5-minute conversations with a human and an LLM: https://arxiv.org/pdf/2503.23674 https://arxiv.org/pdf/2503.23674
- Joeboy 1y ago
  This is kind of an irrelevant (and doubtless unoriginal) shower thought here but, if humans are judging the AI to be human much more often than the human, surely that means the AI is not faithfully reproducing human behaviour.
  - delusional 1y ago
    This comes down to the interpretation of the Turing test. Turing's original test actually pitted the two "unknowns" against each other. Put simply, both the human and the computer would try to make you believe they were the person. The objective of the game was to be seen as human, not to be indistinguishable from human. This is obviously not quite what people understand the Turing test as anymore, and I think that interpretation confusion actually ends up weakening the linked paper. Your thought aptly describes a problem with the paper, but that problem is not present in the Turing test by its original formulation.
    - akoboldfrying 1y ago
      If you're referring to the paper I linked to, their experiments use bona fide 3-party Turing tests as per Turing's original "Imitation Game".
      - delusional 1y ago
        It's hard to say what a "bona fide 3-party Turing test" is. The paper even has a section trying to tackle that issue. I think trying to discuss the minutia of the rules is a path that leads only to madness. The Turing test was always meant to be a philosophical game. The point was to establish a scenario in which a computer could be indistinguishable from a human. Carrying it out in reality in meaningless, unless you're willing to abandon all intuitive morality. Quite frankly, I find the paper you linked misguided. If it was undertaken by some college students, then it's good practice, but if it was carried out by seasoned professionals they should find something better to do.
        akoboldfrying 1y ago
        I only read the abstract, but if the body bears it out and there were decent sample sizes, I think it's interesting and worthwhile. > The Turing test was always meant to be a philosophical game. Says who? Not Alan Turing.
  - akoboldfrying 1y ago
    Sure, a non-human's performance "should" be capped at ~50% for a large sample size. I think seeing a much higher percentage, like 73%, indicates systematic error in the interrogator. This -- the fact that humans are not good at detecting genuine human behaviour -- is really a problem in the Turing test itself, but I don't see a good way to solve it. LLaMa 3.1 with the same prompt "only" managed to be judged human 56% of the time, so perhaps it's actually closer to real human behaviour.
- lostmsu 1y ago
  Shameless self-plug: You can try a two-way variant at https://trashtalk.borg.games/ https://trashtalk.borg.games/ (also have to guess relative ELO) It would be surprising if you won't quickly learn to win.
absummer 1y ago
The original Turing game was about testing for a male of female player. If you want to know more about that, or this research, you could try asking AI for a no-fluff summary. The Transformer architecture and algorithm and matrix multiplication are a bit more involved. It would be hard to keep those inside your chain-of-thought / working memory and still understand what is going on here.
- delusional 1y ago
  > If you want to know more about that, or this research, you could try asking AI for a no-fluff summary. Or I could just read it. With my human eyes. It's like a single page.
  - deleted 1y ago
    [deleted]

akoboldfrying 1y ago

You're right about interactivity, something that I overlooked -- but I think it's nevertheless the case that a large fraction of human interrogators could not distinguish a human from a suitably-system-prompted ChatGPT even over the course of an interactive discussion. ChatGPT 4.5 was judged to be the human 73% of the time in this RCT study, where human interrogators had 5-minute conversations with a human and an LLM: https://arxiv.org/pdf/2503.23674 https://arxiv.org/pdf/2503.23674

Joeboy 1y ago
This is kind of an irrelevant (and doubtless unoriginal) shower thought here but, if humans are judging the AI to be human much more often than the human, surely that means the AI is not faithfully reproducing human behaviour.
- delusional 1y ago
  This comes down to the interpretation of the Turing test. Turing's original test actually pitted the two "unknowns" against each other. Put simply, both the human and the computer would try to make you believe they were the person. The objective of the game was to be seen as human, not to be indistinguishable from human. This is obviously not quite what people understand the Turing test as anymore, and I think that interpretation confusion actually ends up weakening the linked paper. Your thought aptly describes a problem with the paper, but that problem is not present in the Turing test by its original formulation.
  - akoboldfrying 1y ago
    If you're referring to the paper I linked to, their experiments use bona fide 3-party Turing tests as per Turing's original "Imitation Game".
    - delusional 1y ago
      It's hard to say what a "bona fide 3-party Turing test" is. The paper even has a section trying to tackle that issue. I think trying to discuss the minutia of the rules is a path that leads only to madness. The Turing test was always meant to be a philosophical game. The point was to establish a scenario in which a computer could be indistinguishable from a human. Carrying it out in reality in meaningless, unless you're willing to abandon all intuitive morality. Quite frankly, I find the paper you linked misguided. If it was undertaken by some college students, then it's good practice, but if it was carried out by seasoned professionals they should find something better to do.
      - akoboldfrying 1y ago
        I only read the abstract, but if the body bears it out and there were decent sample sizes, I think it's interesting and worthwhile. > The Turing test was always meant to be a philosophical game. Says who? Not Alan Turing.
- akoboldfrying 1y ago
  Sure, a non-human's performance "should" be capped at ~50% for a large sample size. I think seeing a much higher percentage, like 73%, indicates systematic error in the interrogator. This -- the fact that humans are not good at detecting genuine human behaviour -- is really a problem in the Turing test itself, but I don't see a good way to solve it. LLaMa 3.1 with the same prompt "only" managed to be judged human 56% of the time, so perhaps it's actually closer to real human behaviour.
lostmsu 1y ago
Shameless self-plug: You can try a two-way variant at https://trashtalk.borg.games/ https://trashtalk.borg.games/ (also have to guess relative ELO) It would be surprising if you won't quickly learn to win.

absummer 1y ago

The original Turing game was about testing for a male of female player. If you want to know more about that, or this research, you could try asking AI for a no-fluff summary. The Transformer architecture and algorithm and matrix multiplication are a bit more involved. It would be hard to keep those inside your chain-of-thought / working memory and still understand what is going on here.

delusional 1y ago
> If you want to know more about that, or this research, you could try asking AI for a no-fluff summary. Or I could just read it. With my human eyes. It's like a single page.
- deleted 1y ago
  [deleted]