3 ms·

Aside: The Loebner prize is a terrible implementation of a Turing test because the judges have no idea what they're doing. You can't just try to have a normal c

by valtron 13y ago

Aside: The Loebner prize is a terrible implementation of a Turing test because the judges have no idea what they're doing. You can't just try to have a normal conversation and see which one sounds more human. You _have_ to try tricky things like the article mentions; beside syntactically ambiguous sentences, you might also try things that would be obvious to a human but that a computer would easily get confused by unless it was programmed to check for it. Likenotusingspaces. O.r. .i.n.t.e.r.l.e.a.v.i.n.g. .l.e.t.t.e.r.s. .w.i.t.h. .s.o.m.e. .r.a.n.d.o.m. .c.h.a.r.a.c.t.e.r. Also, both the participants being tested (AI and human) should be trying to convince the judge that they're the human and the other is the AI. Given that chatterbots that insert "one-liners" and other non-sequitors do well, I doubt that's happening, so again, the Loebner test is useless.

seiji 13y ago
"Is it an AI" tests based only on text are pretty silly anyway. There are probably a few billion people alive who can't form complete written sentences. If you have a job, you probably encounter dozens of people a year who can't write coherent emails. The Turing Test concept appeared when everybody thought intelligence was mostly words, and we could probably easily make computers spit out meaningful sequences of words. Nobody believes intelligence is how well you can write and read anymore (except fad-based media reports "OMG SIRI IZ INTELLUGENT AI SKYNET!"). The Turing Test concept persists because it's simpler than dirt to explain, even though it has never been a valid method of evaluating anything.
- feral 13y ago
  I don't think you are being fair on the Turing test, or on the intelligence of its creator. In describing the setting of the test, Turing first describes an imitation game where a judge must tell apart a man and a woman. He writes "In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary." So the text format is out of the practicalities of defining a workable test, not because Turing thought 'intelligence is how well you can write and read'. Your other objection is that there are humans who can't form written sentences. Well, they could use the intermediary, if they cant write. But, even if you mean that they can't put a verbal sentence together, the test is framed such that a machine that can pass it can be considered to think, not so that any machine (or human) that can think is supposed to be able to pass the test. Again, from Turing's paper: "May not machines carry out something which ought to be described as thinking but which is very different from what a man does? This objection is a very strong one, but at least we can say that if, nevertheless, a machine can be constructed to play the imitation game satisfactorily, we need not be troubled by this objection." In other words, passing the test - succeeding at the imitation game - is designed to be a sufficient test to demonstrate intelligence, but not a necessary one.
  - seiji 13y ago
    You are quite right. Oddly, I think there's a distinction between the mass-media version of what a Turing Test is and the actual imitation game. Drifting culture impacts what counts as "passing" too. Way Back Then, everything was slightly more formal, proper, and precise. These days, I have SMS chats with some people who rarely reply with more than one or two words (or maybe an emoji if they're feeling really communicative at the moment).
    - dandelany 13y ago
      Sorry, I normally hate nitpicking but since I saw it three times: it's Turing, not Turning.
      - seiji 13y ago
        Wow, I didn't intend that at all (and in all three places too, that's quite a failure). Thanks for noticing. Either my fingers are misbehaving or autocorrect needs a bigger dictionary.
Houshalter 13y ago
The thing about changing the spacing of words isn't really fair on the computer. It'd be like spelling words phonetically and expecting a deaf person to understand it. Or if I gave you text in binary and expected you to understand quickly. The point is to prove that the computer is actually understanding what you are saying in normal English. It's not necessary for it to be able to figure out different formats that words can be written, in order to prove that it's intelligent. Also it's a terrible implementation of the Turing test on purpose. No one has ever come close to real AI that could pass a real Turing test. The point of the prize is more of who can come up with the most convincing chatbots. At least if I understand it correctly.
- tomp 13y ago
  That would be a very useful test of telling deaf and hearing people appart.

by valtron 13y ago

seiji 13y ago

"Is it an AI" tests based only on text are pretty silly anyway. There are probably a few billion people alive who can't form complete written sentences. If you have a job, you probably encounter dozens of people a year who can't write coherent emails. The Turing Test concept appeared when everybody thought intelligence was mostly words, and we could probably easily make computers spit out meaningful sequences of words. Nobody believes intelligence is how well you can write and read anymore (except fad-based media reports "OMG SIRI IZ INTELLUGENT AI SKYNET!"). The Turing Test concept persists because it's simpler than dirt to explain, even though it has never been a valid method of evaluating anything.

feral 13y ago
I don't think you are being fair on the Turing test, or on the intelligence of its creator. In describing the setting of the test, Turing first describes an imitation game where a judge must tell apart a man and a woman. He writes "In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary." So the text format is out of the practicalities of defining a workable test, not because Turing thought 'intelligence is how well you can write and read'. Your other objection is that there are humans who can't form written sentences. Well, they could use the intermediary, if they cant write. But, even if you mean that they can't put a verbal sentence together, the test is framed such that a machine that can pass it can be considered to think, not so that any machine (or human) that can think is supposed to be able to pass the test. Again, from Turing's paper: "May not machines carry out something which ought to be described as thinking but which is very different from what a man does? This objection is a very strong one, but at least we can say that if, nevertheless, a machine can be constructed to play the imitation game satisfactorily, we need not be troubled by this objection." In other words, passing the test - succeeding at the imitation game - is designed to be a sufficient test to demonstrate intelligence, but not a necessary one.
- seiji 13y ago
  You are quite right. Oddly, I think there's a distinction between the mass-media version of what a Turing Test is and the actual imitation game. Drifting culture impacts what counts as "passing" too. Way Back Then, everything was slightly more formal, proper, and precise. These days, I have SMS chats with some people who rarely reply with more than one or two words (or maybe an emoji if they're feeling really communicative at the moment).
  - dandelany 13y ago
    Sorry, I normally hate nitpicking but since I saw it three times: it's Turing, not Turning.
    - seiji 13y ago
      Wow, I didn't intend that at all (and in all three places too, that's quite a failure). Thanks for noticing. Either my fingers are misbehaving or autocorrect needs a bigger dictionary.

Houshalter 13y ago

The thing about changing the spacing of words isn't really fair on the computer. It'd be like spelling words phonetically and expecting a deaf person to understand it. Or if I gave you text in binary and expected you to understand quickly. The point is to prove that the computer is actually understanding what you are saying in normal English. It's not necessary for it to be able to figure out different formats that words can be written, in order to prove that it's intelligent. Also it's a terrible implementation of the Turing test on purpose. No one has ever come close to real AI that could pass a real Turing test. The point of the prize is more of who can come up with the most convincing chatbots. At least if I understand it correctly.

tomp 13y ago
That would be a very useful test of telling deaf and hearing people appart.