4 ms·

That's the part of the Turing Test I've never understood - it seems very dependent on the skill and intelligence of the human tester. Did Turing talk about thi

That's the part of the Turing Test I've never understood - it seems very dependent on the skill and intelligence of the human tester. Did Turing talk about this aspect? I seem to remember there was supposed to be a panel or committee rather than a single individual?

nurettin 6y ago
Turing explains that the machine could not give an "appropriately meaningful" answer to whatever is said in it's presence, which the "dullest of men" could do. So the basic quality he seeks in a machine that could pass the so called turing test is a machine that produces meaning. That meaning comes from any mind that is intact or coherent. So unless we take the leap from making category deciders and inference machines to actually building minds, we won't get anywhere near his definition of a machine that is truly intelligent.
- zwischenzug 6y ago
  Is this a real comment, or produced by an AI? I couldn't make sense of it. This is a genuine question.
  - oldsj 6y ago
    I think the cat’s out of the bag now and we really won’t know if we’re reading or responding to an actual human anymore.
    - gear54rus 6y ago
      Reading comments on the internet has nothing to do with the test tho. You have to be able to query the agent.
      - andybak 6y ago
        > You have to be able to query the agent. That's what the "reply" button is for.
  - Don_Patrick 6y ago
    I know people who talk like this, and the user history checks out. Either way, the comment is misinforming: Those quotes are from Descartes, not Turing, whose views as to whether machines could think were opposite. https://plato.stanford.edu/entries/turing-test https://plato.stanford.edu/entries/turing-test
    - zwischenzug 6y ago
      Right: the attribution to Turing felt completely wrong.
  - zaroth 6y ago
    The use of “quoted words” is strong evidence of GPT. It’s very fond of that rhetorical technique.
    - rosstex 6y ago
      I'm not sure, my PhD advisor also uses "quoted words" very frequently in her writing. In GPT-3's defense, she is a prolific writer :)
  - nurettin 6y ago
    The whole thing was generated. Any specific reason you couldn't make sense of it? Any non-sequiturs? Would you prefer more monty python references?
    - zwischenzug 6y ago
      I honestly can't tell if this was generated or not either. Where does the monty python reference come from? FWIW: - The ab nihilo mention of 'Turing explains' (explains what? where?) feels odd. - The quotes are redundant, but seem designed to make the statement sound weighty. - "It's" is a common grammatical error in corpuses, but not one likely to be made by any kind of practising philosopher. - The concept of "meaning" is one implicitly eschewed by the Turing test, so the discussion of meaning here seems odd. - The comment is circular: unless we build minds, we won't build things that pass the Turing tests, which are minds.
      - nurettin 6y ago
        Well, I will dismiss all the nitpicky npc nonsense here and just repeat myself. Are we trying to build minds? Or is all the effort going into building automated money savers that take away trivial jobs? That is the main point here.
Don_Patrick 6y ago
Turing only proposed his game as a hypothetical argument, so he barely specified any rules on how to actually perform such a test in practice. He referred to the judging party either as "the man" or "the average interrogator", he never specified a number. Official Turing test events have had anywhere between four to hundreds of people judging.
qayxc 6y ago
The test originated from the Imitation Game. Turing envisioned a game in which two people (A & B) are tasked to pretent to be the other (not at the same time, though). An interviewer is then trying to decide who of them is actually the one they pretend to be by asking questions. In order to remove any hints from appearance, voice, or handwriting, a typewriter is used for interacting. For example A could be male and B could be female and A is asked to pretend to be B. The interviewer can then ask questions to decide whether A is male or female (yes, I am fully aware that in 2020 this: https://youtu.be/z2_8cfVpXbo?t=129 https://youtu.be/z2_8cfVpXbo?t=129 could also happen). Turing then proposed to replace A or B with a computer instead and ask the interviewer to decide which of them is the human. In this scenario, do you really think the interviewer would bombard the candidates with trivia questions and logic puzzles to find out? The idea was, that other aspects of human intelligence, like a sense for beauty, poetry, etc. would be used instead to differentiate between the two. Questions like "Do you enjoy reading Harry Potter?" and depending on the answer you could further ask why or why not the subject likes or dislikes the books. This would be much more insightful and coming up with such questions doesn't require any particular skill on part of the interviewer. You can even get tricky and return to the topic after talking about something else to see whether you get a similar response or to test the subject's memory.
- andybak 6y ago
  That's exactly why I said "it seems very dependent on the skill and intelligence of the human tester." A test "pass" or "fail" could potentially be the fault of the tester as much as it is a sign of intelligence in the AI. How do you evaluate how capable someone is of administering a worthwhile Turing Test? Maybe they should be tested beforehand. I propose a system involving a remote teletype machine and human tester...
  - qayxc 6y ago
    The test itself doesn't rely on one person, though. It's a statistical measure, so for each "game" you have a number of interviewers/judges who test the system. If the system has any of them fooled, the test can be considered "passed". AFAIK there's no precise number attached to this, so it's not like ">x% people fooled => pass". So in essence it's not "the human tester" but "the average human tester" for some arbitrary definition of "average". It's a really interesting dilemma if you think about it: is a forger good if they're able to fool everyone but experts? Does an AI pass the Turing test if it fools the general public, but every AI expert who knows the system would be able to tell after just a few sentences?

nurettin 6y ago

Turing explains that the machine could not give an "appropriately meaningful" answer to whatever is said in it's presence, which the "dullest of men" could do. So the basic quality he seeks in a machine that could pass the so called turing test is a machine that produces meaning. That meaning comes from any mind that is intact or coherent. So unless we take the leap from making category deciders and inference machines to actually building minds, we won't get anywhere near his definition of a machine that is truly intelligent.

zwischenzug 6y ago
Is this a real comment, or produced by an AI? I couldn't make sense of it. This is a genuine question.
- oldsj 6y ago
  I think the cat’s out of the bag now and we really won’t know if we’re reading or responding to an actual human anymore.
  - gear54rus 6y ago
    Reading comments on the internet has nothing to do with the test tho. You have to be able to query the agent.
    - andybak 6y ago
      > You have to be able to query the agent. That's what the "reply" button is for.
- Don_Patrick 6y ago
  I know people who talk like this, and the user history checks out. Either way, the comment is misinforming: Those quotes are from Descartes, not Turing, whose views as to whether machines could think were opposite. https://plato.stanford.edu/entries/turing-test https://plato.stanford.edu/entries/turing-test
  - zwischenzug 6y ago
    Right: the attribution to Turing felt completely wrong.
- zaroth 6y ago
  The use of “quoted words” is strong evidence of GPT. It’s very fond of that rhetorical technique.
  - rosstex 6y ago
    I'm not sure, my PhD advisor also uses "quoted words" very frequently in her writing. In GPT-3's defense, she is a prolific writer :)
- nurettin 6y ago
  The whole thing was generated. Any specific reason you couldn't make sense of it? Any non-sequiturs? Would you prefer more monty python references?
  - zwischenzug 6y ago
    I honestly can't tell if this was generated or not either. Where does the monty python reference come from? FWIW: - The ab nihilo mention of 'Turing explains' (explains what? where?) feels odd. - The quotes are redundant, but seem designed to make the statement sound weighty. - "It's" is a common grammatical error in corpuses, but not one likely to be made by any kind of practising philosopher. - The concept of "meaning" is one implicitly eschewed by the Turing test, so the discussion of meaning here seems odd. - The comment is circular: unless we build minds, we won't build things that pass the Turing tests, which are minds.
    - nurettin 6y ago
      Well, I will dismiss all the nitpicky npc nonsense here and just repeat myself. Are we trying to build minds? Or is all the effort going into building automated money savers that take away trivial jobs? That is the main point here.

Don_Patrick 6y ago

Turing only proposed his game as a hypothetical argument, so he barely specified any rules on how to actually perform such a test in practice. He referred to the judging party either as "the man" or "the average interrogator", he never specified a number. Official Turing test events have had anywhere between four to hundreds of people judging.

qayxc 6y ago

The test originated from the Imitation Game. Turing envisioned a game in which two people (A & B) are tasked to pretent to be the other (not at the same time, though). An interviewer is then trying to decide who of them is actually the one they pretend to be by asking questions. In order to remove any hints from appearance, voice, or handwriting, a typewriter is used for interacting. For example A could be male and B could be female and A is asked to pretend to be B. The interviewer can then ask questions to decide whether A is male or female (yes, I am fully aware that in 2020 this: https://youtu.be/z2_8cfVpXbo?t=129 https://youtu.be/z2_8cfVpXbo?t=129 could also happen). Turing then proposed to replace A or B with a computer instead and ask the interviewer to decide which of them is the human. In this scenario, do you really think the interviewer would bombard the candidates with trivia questions and logic puzzles to find out? The idea was, that other aspects of human intelligence, like a sense for beauty, poetry, etc. would be used instead to differentiate between the two. Questions like "Do you enjoy reading Harry Potter?" and depending on the answer you could further ask why or why not the subject likes or dislikes the books. This would be much more insightful and coming up with such questions doesn't require any particular skill on part of the interviewer. You can even get tricky and return to the topic after talking about something else to see whether you get a similar response or to test the subject's memory.

andybak 6y ago
That's exactly why I said "it seems very dependent on the skill and intelligence of the human tester." A test "pass" or "fail" could potentially be the fault of the tester as much as it is a sign of intelligence in the AI. How do you evaluate how capable someone is of administering a worthwhile Turing Test? Maybe they should be tested beforehand. I propose a system involving a remote teletype machine and human tester...
- qayxc 6y ago
  The test itself doesn't rely on one person, though. It's a statistical measure, so for each "game" you have a number of interviewers/judges who test the system. If the system has any of them fooled, the test can be considered "passed". AFAIK there's no precise number attached to this, so it's not like ">x% people fooled => pass". So in essence it's not "the human tester" but "the average human tester" for some arbitrary definition of "average". It's a really interesting dilemma if you think about it: is a forger good if they're able to fool everyone but experts? Does an AI pass the Turing test if it fools the general public, but every AI expert who knows the system would be able to tell after just a few sentences?