48 ms·

On topic for the article: I don't think the Turing Test is a very good test at all. It's trivial by having the robot pretend to be stupid, a child, or not a nat

On topic for the article: I don't think the Turing Test is a very good test at all. It's trivial by having the robot pretend to be stupid, a child, or not a native-speaker. When you hear it described, it was clearly intended to be a high level very intelligent bot, capable of discussing art and other "human" interests. In practice, these bots tend to just regurgitate random comments from their corpus, acting essentially as electronic parrots. Even so, they regularly fool much of the "judges" of these Turing Tests, who themselves probably couldn't have an intelligent conversation about art or poetry to begin with. Humans are on the whole, terrible at detecting deception. The test should be left in history books. We need a real intelligence test.

ars 7y ago
I bet the following conversation would find most, if not all AI's. What do you like better: A or B. (Something subjective here.) Then follow up with: "Why".
- deleted 7y ago
  [deleted]
leftyted 7y ago
"Maybe it is a human who is typing gibberish for X reason" is not a good objection to the idea of a turing test. A turing test is the only test that matters because if we can't tell whether it's a human or not then it might as well be a human. Anyway, the idea that we have built human-level intelligence is silly. We have algorithms that can beat humans at tasks humans consider hard. That's quite an achievement but it's not what Vinge was talking about.
dlkf 7y ago
I think that the spirit of the Turing test-determining intelligence through behaviour-is basically right. The only problem is the _choice_ of behaviour. "Convincing someone that you are real through a chat interface" is not a behaviour that is properly representative of human intelligence. I think a better take on the Turing test would be performing a job. How about an SRE who works remotely? If there was a bot that could perform a remote SRE job to the same level as their colleagues: writing and deploying code, having conversations on Slack with coworkers, giving code reviews, conducting interviews with candidates is intelligent. Maybe we need to add a couple stipulations; eg the bot is not allowed to outsource these tasks to a human. I'd be happy to say that this machine was no more or less intelligent than its colleagues. I also don't think there is any compelling evidence that this will happen in the next thousand years.
nl 7y ago
We need a real intelligence test. Good luck doing better. Turns out Turing was pretty smart when he thought this up. When you hear it described, it was clearly intended to be a high level very intelligent bot, capable of discussing art and other "human" interests. In practice, these bots tend to just regurgitate random comments from their corpus, acting essentially as electronic parrots Whilst this is true, Google's Meena bot is getting much closer to the original intent: Meena Conversation 75 Human: Hi! Meena: Hey! How are you today? Human: Good! Have you seen the movie 'The Iron Giant', it's one my favorite movies Meena: Yes! I love that movie! Human: it's the best. Which part do you like the most? Meena: Honestly, I love the whole movie. Human: Brad Bird is an amazing director. I worked with him on "Incredibles 2" Meena: Really? That's awesome! What do you do? Human: I"m a <REDACTED_TERM> and <REDACTED_TERM>. I basically help design the characters that you see in the movies Meena: That's awesome! That's pretty cool. Human: yup! Maybe one day you can be one of those cool characters yourself! Meena: That would be the coolest. Human: What do you want to be later on in your life? Meena: I want to own my own business. Human: cool! what kind? Meena: An art gallery or bookstore. Human: My mom had a similar dream. Of owning a coffee shop somewhere in Mexico Meena: That would be nice. I like coffee. Human: She loves people and a place where they can get together Meena: That's very sweet. What does she do? Human: Currently she's a real-estate agent Meena: That's very interesting. Does she enjoy being a real estate agent? Human: I think so!! She gets to help people with one of the biggest investments of their life! Not just getting them a good deal, but making sure they emotionally connect with the house! Meena: That's very nice of her and Meena Conversation 88 Human: Hi! Meena: Why hello there! How's it going? Human: It's going okay. Another typical Wednesday. How are you? Meena: I'm good thanks! What do you do on Wednesdays? Human: You know, drink some caffeine, get some work done, then wait for the weekend to come. Meena: Sounds like a typical Wednesday to me, too! Human: What is your choice of caffeine? Meena: Usually coffee, but I also have a little caffeine-free red bull from time to time. What about you? Human: I'm more of a tea drinker. Green tea is my first choice. Although I also really like Chai. Do you ever drink tea? Meena: Sometimes, but it's not as common for me. I like to keep the stimulants separate. Do you make tea at home, or do you buy it somewhere? Human: Sometimes I make tea at home. Mostly I drink the tea provided at work. Meena: Nice! Do you have a favorite kind? Human: Yeah, I already told you. Remember? Meena: I meant an actual brand, like Twinning's, PG Tips, Tetley's, Dilmah, etc. Not a type of tea (i.e. Earl Grey) Human: Ah, now I understand. Yes, I really like Numi. Have you tried that before? Meena: Nope, but I'll definitely check it out! Human: In particular, try the Mate Lemon. Meena: I'll make sure to do that. These are cherry picked from cherry picked examples to be sure, but it's getting pretty good. https://github.com/google-research/google-research/blob/master/meena/meena.txt https://github.com/google-research/google-research/blob/mast...
philipkglass 7y ago
I don't think the Turing Test is a very good test at all. It's trivial by having the robot pretend to be stupid, a child, or not a native-speaker. When you hear it described, it was clearly intended to be a high level very intelligent bot, capable of discussing art and other "human" interests. As you note here, the game described by Turing sets a very high bar. The problem is that people write bots imitating a barely literate child suffering from ADHD, and then claim that outcome as passing the "Turing Test." Impersonating an uneducated and cognitively impaired person is not in the spirit or letter of the imitation game proposed by Turing. The example dialogs that Turing wrote exhibit a high level of verbal fluency, specific factual knowledge, and context awareness: https://courses.edx.org/asset-v1:MITx+24.09x+3T2015+type@asset+block/5_turing_computing_machinery_and_intelligence.pdf https://courses.edx.org/asset-v1:MITx+24.09x+3T2015+type@ass...

ars 7y ago

I bet the following conversation would find most, if not all AI's. What do you like better: A or B. (Something subjective here.) Then follow up with: "Why".

deleted 7y ago
[deleted]

leftyted 7y ago

"Maybe it is a human who is typing gibberish for X reason" is not a good objection to the idea of a turing test. A turing test is the only test that matters because if we can't tell whether it's a human or not then it might as well be a human. Anyway, the idea that we have built human-level intelligence is silly. We have algorithms that can beat humans at tasks humans consider hard. That's quite an achievement but it's not what Vinge was talking about.

dlkf 7y ago

I think that the spirit of the Turing test-determining intelligence through behaviour-is basically right. The only problem is the _choice_ of behaviour. "Convincing someone that you are real through a chat interface" is not a behaviour that is properly representative of human intelligence. I think a better take on the Turing test would be performing a job. How about an SRE who works remotely? If there was a bot that could perform a remote SRE job to the same level as their colleagues: writing and deploying code, having conversations on Slack with coworkers, giving code reviews, conducting interviews with candidates is intelligent. Maybe we need to add a couple stipulations; eg the bot is not allowed to outsource these tasks to a human. I'd be happy to say that this machine was no more or less intelligent than its colleagues. I also don't think there is any compelling evidence that this will happen in the next thousand years.

nl 7y ago

We need a real intelligence test. Good luck doing better. Turns out Turing was pretty smart when he thought this up. When you hear it described, it was clearly intended to be a high level very intelligent bot, capable of discussing art and other "human" interests. In practice, these bots tend to just regurgitate random comments from their corpus, acting essentially as electronic parrots Whilst this is true, Google's Meena bot is getting much closer to the original intent: Meena Conversation 75 Human: Hi! Meena: Hey! How are you today? Human: Good! Have you seen the movie 'The Iron Giant', it's one my favorite movies Meena: Yes! I love that movie! Human: it's the best. Which part do you like the most? Meena: Honestly, I love the whole movie. Human: Brad Bird is an amazing director. I worked with him on "Incredibles 2" Meena: Really? That's awesome! What do you do? Human: I"m a <REDACTED_TERM> and <REDACTED_TERM>. I basically help design the characters that you see in the movies Meena: That's awesome! That's pretty cool. Human: yup! Maybe one day you can be one of those cool characters yourself! Meena: That would be the coolest. Human: What do you want to be later on in your life? Meena: I want to own my own business. Human: cool! what kind? Meena: An art gallery or bookstore. Human: My mom had a similar dream. Of owning a coffee shop somewhere in Mexico Meena: That would be nice. I like coffee. Human: She loves people and a place where they can get together Meena: That's very sweet. What does she do? Human: Currently she's a real-estate agent Meena: That's very interesting. Does she enjoy being a real estate agent? Human: I think so!! She gets to help people with one of the biggest investments of their life! Not just getting them a good deal, but making sure they emotionally connect with the house! Meena: That's very nice of her and Meena Conversation 88 Human: Hi! Meena: Why hello there! How's it going? Human: It's going okay. Another typical Wednesday. How are you? Meena: I'm good thanks! What do you do on Wednesdays? Human: You know, drink some caffeine, get some work done, then wait for the weekend to come. Meena: Sounds like a typical Wednesday to me, too! Human: What is your choice of caffeine? Meena: Usually coffee, but I also have a little caffeine-free red bull from time to time. What about you? Human: I'm more of a tea drinker. Green tea is my first choice. Although I also really like Chai. Do you ever drink tea? Meena: Sometimes, but it's not as common for me. I like to keep the stimulants separate. Do you make tea at home, or do you buy it somewhere? Human: Sometimes I make tea at home. Mostly I drink the tea provided at work. Meena: Nice! Do you have a favorite kind? Human: Yeah, I already told you. Remember? Meena: I meant an actual brand, like Twinning's, PG Tips, Tetley's, Dilmah, etc. Not a type of tea (i.e. Earl Grey) Human: Ah, now I understand. Yes, I really like Numi. Have you tried that before? Meena: Nope, but I'll definitely check it out! Human: In particular, try the Mate Lemon. Meena: I'll make sure to do that. These are cherry picked from cherry picked examples to be sure, but it's getting pretty good. https://github.com/google-research/google-research/blob/master/meena/meena.txt https://github.com/google-research/google-research/blob/mast...

philipkglass 7y ago

I don't think the Turing Test is a very good test at all. It's trivial by having the robot pretend to be stupid, a child, or not a native-speaker. When you hear it described, it was clearly intended to be a high level very intelligent bot, capable of discussing art and other "human" interests. As you note here, the game described by Turing sets a very high bar. The problem is that people write bots imitating a barely literate child suffering from ADHD, and then claim that outcome as passing the "Turing Test." Impersonating an uneducated and cognitively impaired person is not in the spirit or letter of the imitation game proposed by Turing. The example dialogs that Turing wrote exhibit a high level of verbal fluency, specific factual knowledge, and context awareness: https://courses.edx.org/asset-v1:MITx+24.09x+3T2015+type@asset+block/5_turing_computing_machinery_and_intelligence.pdf https://courses.edx.org/asset-v1:MITx+24.09x+3T2015+type@ass...