3 ms·

The Turing test is to our understanding of intelligence what sleight of hand is to our understanding of physics. Tricking people, as a goal, is not conductive t

The Turing test is to our understanding of intelligence what sleight of hand is to our understanding of physics. Tricking people, as a goal, is not conductive to science. A researcher claiming to have passed the Turing test instantly discredits himself as a prestidigitator looking for PR buzz. The present article is a textbook example of this. As a side note, if you are focusing on disembodied, language-based human-like intelligence, then the paradigm you operate in is many decades behind. The Turing test was conceived at a time when the notion of thinking machines had just started to emerge --a very different time from today, where we have 60 years of AI research behind us. The Turing test has been irrelevant for longer than most AI researchers have been alive. I have never seen it used for anything else than smoke-and-mirrors PR operations.

acchow 12y ago
Your opinion is not universally accepted. (Perhaps not even mainstream?). There's an element of behaviorism which stretches back to Descartes - how can you know that I exist? That I think? You can only observe through my behavior; that my behavior mimics yours. How then can we judge machines any other way?
- VLM 12y ago
  Your spec is over generalized into all forms of observation and comparison. The Turing test is very limited into written 1 on 1 conversation as culturally conceived by humans. There's no particular reason to assume a non-human intelligence would have a human-like culture of communication, which kind of breaks it. Here's a fun idea for several intelligence tests we can call the VLM tests that have nothing to do with two way conversation like the Turing test. Given "a machine intelligence" spin up a couple million of them in a "fun" simulation environment and see how much thermodynamic dis-equilibrium they generate by whatever social interaction they see fit to apply to each other. Is it as interesting (aka thermodynamic dis-equilibrium) as a GoL or a real world anthill or a Dwarf Fortress or a Google Earth? Is their simulated culture as interesting to read as HN, or as dumb as youtube comments (which used to be the gold standard of dumbness in social media) Assuming you can crack the literary code (if any exists) another game to play is extract the meme-flow of a culture of AI vs a culture of 4chan and vote for whichever meme came from a more intelligent group. This is Turing-ish WRT human observers majority vote and such, but is completely non-interactive, merely humans, or even trained sociologists, trying to figure out given two memes which is more intelligent. Getting out the Sherlock Holmes hat, its possible to determine if an artifact came from an intelligence without talking to the intelligence for awhile. I suspect archeologists have really fun debates on this topic. Is this a stone hammer or merely a peculiar river rock, etc.
- fchollet 12y ago
  The Turing test only tests for mimicry in human conversation, not "intelligence" as measured through a behavioral framework. A better "Turing test" proposed a few years ago was, "develop a team of robots that play soccer so well they can win the World Cup". While not the best possible driver of AI research, the quest for such a goal would still drive AI research orders or magnitude better than "trick random people to mistake a chatbot for a human".
  - dayv 12y ago
    Though how accurate the Turing test correlates to "intelligence" can be debated, Turing himself believed that mastery of language demonstrates reasonable intelligence. However, in this case, "Eugene" claimed English as his second language, which seems as close to cheating as it gets.
  - xerophtye 12y ago
    >develop a team of robots that play soccer so well they can win the World Cup How is this any better that "build a bot that can beat Chess Masters" ?? Well, apart from the part where this includes multiple agents, and swarm robotics, etc. But frankly, setting a game as a bar actually covers relatively less scope in my opinion. Conversational bot actually does cover a LOT of scope if you think about all the possible ways the conversation can be taken. In fact, we as humans use conversation to judge other humans' intelligence as well. E.g. job interviews. Though perhaps an advanced turing test could include performance in a variety of social situations like... - convince employers to hire you, - Convince a girl to go out with you - Convince a customer to buy something - Debates... (Presidential debates by AI would be interesting)
  - TeMPOraL 12y ago
    Is it really better? It sounds completely within capabilities of current robotics and algorithms. Mix up DARPA robots and AI from FIFA and you'll likely have a winning team.
    - fchollet 12y ago
      Please. That is many decades away, and the hard part about it is definitely not the soccer strategy AI...
- andreasvc 12y ago
  His opinion is actually a mainstream opinion in AI, see for example the textbook by Russell & Norvig. Descartes was the very opposite of a behaviorist, namely a rationalist. I do agree that we don't have anything that is clearly better than the Turing test, but this just goes to show how far we still have to go.
  - mcguire 12y ago
    I missed the AI Winter. Norvig survived it. I would imagine that colors his thoughts on work towards general intelligence. Also the fact that all AI research that has actually done anything interesting has been of the "weak" (as I learned to call it) variety. But look at me, ascribing thoughts and intentions on a black box that I cannot be sure really exists.
mcguire 12y ago
How do you define intelligence, then?

acchow 12y ago

Your opinion is not universally accepted. (Perhaps not even mainstream?). There's an element of behaviorism which stretches back to Descartes - how can you know that I exist? That I think? You can only observe through my behavior; that my behavior mimics yours. How then can we judge machines any other way?

VLM 12y ago
Your spec is over generalized into all forms of observation and comparison. The Turing test is very limited into written 1 on 1 conversation as culturally conceived by humans. There's no particular reason to assume a non-human intelligence would have a human-like culture of communication, which kind of breaks it. Here's a fun idea for several intelligence tests we can call the VLM tests that have nothing to do with two way conversation like the Turing test. Given "a machine intelligence" spin up a couple million of them in a "fun" simulation environment and see how much thermodynamic dis-equilibrium they generate by whatever social interaction they see fit to apply to each other. Is it as interesting (aka thermodynamic dis-equilibrium) as a GoL or a real world anthill or a Dwarf Fortress or a Google Earth? Is their simulated culture as interesting to read as HN, or as dumb as youtube comments (which used to be the gold standard of dumbness in social media) Assuming you can crack the literary code (if any exists) another game to play is extract the meme-flow of a culture of AI vs a culture of 4chan and vote for whichever meme came from a more intelligent group. This is Turing-ish WRT human observers majority vote and such, but is completely non-interactive, merely humans, or even trained sociologists, trying to figure out given two memes which is more intelligent. Getting out the Sherlock Holmes hat, its possible to determine if an artifact came from an intelligence without talking to the intelligence for awhile. I suspect archeologists have really fun debates on this topic. Is this a stone hammer or merely a peculiar river rock, etc.
fchollet 12y ago
The Turing test only tests for mimicry in human conversation, not "intelligence" as measured through a behavioral framework. A better "Turing test" proposed a few years ago was, "develop a team of robots that play soccer so well they can win the World Cup". While not the best possible driver of AI research, the quest for such a goal would still drive AI research orders or magnitude better than "trick random people to mistake a chatbot for a human".
- dayv 12y ago
  Though how accurate the Turing test correlates to "intelligence" can be debated, Turing himself believed that mastery of language demonstrates reasonable intelligence. However, in this case, "Eugene" claimed English as his second language, which seems as close to cheating as it gets.
- xerophtye 12y ago
  >develop a team of robots that play soccer so well they can win the World Cup How is this any better that "build a bot that can beat Chess Masters" ?? Well, apart from the part where this includes multiple agents, and swarm robotics, etc. But frankly, setting a game as a bar actually covers relatively less scope in my opinion. Conversational bot actually does cover a LOT of scope if you think about all the possible ways the conversation can be taken. In fact, we as humans use conversation to judge other humans' intelligence as well. E.g. job interviews. Though perhaps an advanced turing test could include performance in a variety of social situations like... - convince employers to hire you, - Convince a girl to go out with you - Convince a customer to buy something - Debates... (Presidential debates by AI would be interesting)
- TeMPOraL 12y ago
  Is it really better? It sounds completely within capabilities of current robotics and algorithms. Mix up DARPA robots and AI from FIFA and you'll likely have a winning team.
  - fchollet 12y ago
    Please. That is many decades away, and the hard part about it is definitely not the soccer strategy AI...
andreasvc 12y ago
His opinion is actually a mainstream opinion in AI, see for example the textbook by Russell & Norvig. Descartes was the very opposite of a behaviorist, namely a rationalist. I do agree that we don't have anything that is clearly better than the Turing test, but this just goes to show how far we still have to go.
- mcguire 12y ago
  I missed the AI Winter. Norvig survived it. I would imagine that colors his thoughts on work towards general intelligence. Also the fact that all AI research that has actually done anything interesting has been of the "weak" (as I learned to call it) variety. But look at me, ascribing thoughts and intentions on a black box that I cannot be sure really exists.

mcguire 12y ago

How do you define intelligence, then?