7 ms·

Looking forward to listening to this recording; I'm always telling people interested in the philosphy of AI to read Turing's papers, as, like the author says, t

by feral 9y ago

Looking forward to listening to this recording; I'm always telling people interested in the philosphy of AI to read Turing's papers, as, like the author says, they are surprisingly relevant. Have to comment on this though: " Turing then makes one firm prediction, that by the end of the 20th century computers would be able to answer questions in a manner indistinguishable from a human being – this is the famous Turing test. Turing’s prediction may have been a couple of decades early, but with the rise of digital assistants I would have to say he was completely right. " The bit I've italicized is nonsense in an article by someone studying Turing's work. The Turing test is clearly designed as a pragmatic definition of a sufficient test for AI, and assumes a sophisticated human judge. The idea that just because we are building digital assistants we are some how close to passing the Turing test and making genuinely thinking machines doesn't hold up at all. I don't think that argument would have impressed Turing, and I'm sure he'd rather we say his prediction was wrong than introduce confusion here. We have so much confusion on this issue, which Turing tried so hard to dispel, it's a pity to see more.

blauditore 9y ago
Yeah, I think digital assistants have not been built to pass a turing test, i.e. to have full human communication skills. They've just been marketed that way.
GolDDranks 9y ago
Indeed, the virtue of the Turing test is that it's a very general framework – the freedom of the framework allows us to test the AI using any communicational method at hand. One of such is Winograd Schemas [1] – a test of commonsensical reasoning, in which the current conversational AIs spectacularly fail at. [1] https://cs.nyu.edu/faculty/davise/papers/WinogradSchemas/WS.html https://cs.nyu.edu/faculty/davise/papers/WinogradSchemas/WS....
- GolDDranks 9y ago
  Btw. here is a sketch of an embedding like that: [Session starts] <Challenger> Hi. <Judge> Hi. Now, listen to me. My job here is to judge whether you are a human or an AI. I also know that your job is to try and persuade me to believe you are a human, no matter which one you really are. I happen to have a few questions that are easy for humans and hard for AIs. So if you are a human, answering them shouldn't be a problem to you, but if you avoid answering, I'm going to judge that you are an AI, no excuses. Here is the first question: ... [A pattern of Winograd schema challenges ensues.] So as we can see, the Turing test doesn't have to be any random chitchat nor it doesn't have to pretend to be "normal human-to-human-like discussion", as the Judge has the ability to participate and steer the conversation and zero in on the areas that are most likely to expose the AI. However, if the human participants don't have any penalty from being labeled as AI, this doesn't obviously work, as they don't have any pressure to prove anything. So this kind of an embedding wouldn't work with "imitation game" -like rules, but it would work with "prove your human-ness" -like rules.
  - Retric 9y ago
    If you classify say 50% of people as AI's and 50% of AI's as people then those AI's passed the Turing test. So, you are limited to questions that most people will answer correctly. Further, if you find some unusual question that works today someone can just add it to the program for next time.
    - GolDDranks 9y ago
      Did you check the link about the Winograd schema challenge? The questions test common sense reasoning, and are very easy for humans to answer. An example: The trophy doesn't fit into the brown suitcase because it's too large. What is too large? A: The trophy B: The suitcase
      - Retric 9y ago
        Coding a specific solution to questions in exactly that format is not inherently difficult. If you keep writing specific solutions to this class of questions it's eventually had to come up with a simple question that's still unknown. Further you can't get around that by making ever more complex questions because humans will eventually start messing them up.
    - edmccard 9y ago
      >If you classify say 50% of people as AI's and 50% of AI's as people then those AI's passed the Turing test. And if your version of the test is "heads it's an AI, tails it's human" then any AI's that are classified as human will have "passed the Turing test."
      - Retric 9y ago
        I don't think you understood. The original test specifically had exactly one human and one AI. So, if the judge is forced to do a coin flip that really is success. If the judge does a coin flip because they are lazy then that's not a Turning test.
  - red75prime 9y ago
marzell 9y ago
I feel we could have reached that point by now, for instance is there was a Turing Race a la the Space Race. The driving factor though is we don't specifically need that from technology. Even when we get to that point, there's tons of reasons we will often want to easily distinguish whether we are communicating with someone vs something. Being distinguishable doesn't mean an AI is any less effective or easy to communicate with. And who knows, perhaps this tech is already in the wild - by definition wouldn't it be fooling us?
Retric 9y ago
Computers have passed limited versions of the Turing test, as in one person was mistakenly classified as AI and an AI classified as a person. Now, I am not saying we could have an expert chat with one for an hour on any topic and still be fooled, but people have already chatted with an AI without noticing it was an AI an not a person. AGI is a different problem. But, faking AGI to the point someone can't tell is more of a philosophical question than a failure of the Turing test.
- olavk 9y ago
  people have already chatted with an AI without noticing it was an AI an not a person I'm sure this have worked in some limited context where the human was not actively trying to figure out if it was a human or AI. I mean, I have sometimes at a distance mistaken a mannequin for a human - that does not mean mannequins are now indistinguishable from humans. In the Turing tests the judge is active trying to discern if they are talking to a (adult, literate, mentally sound) human or an AI. AFAIK no AI have ever passed this. (Of course it is easy for a human to impersonate a bad AI by answering erratically, but that doesn't really prove anything.)
  - JoeAltmaier 9y ago
    I'm not sure that masquerading as a normal human being is any kind of a bar for an AI to pass. How about, passing for an expert in a field e.g. medicine, investing, navigation? I'd much rather see my AI certified that way.
    - olavk 9y ago
      Such a specialized AI would probably be more useful in practice, but that is not really what the Turing test is concerned with. The Turing test is a behaviorist approach to deciding the question if a computer "can think". In any case, it think it is much easier to create an AI specialized in a particular domain rather than a general "human-like" AI.
      - JoeAltmaier 9y ago

by feral 9y ago

blauditore 9y ago

Yeah, I think digital assistants have not been built to pass a turing test, i.e. to have full human communication skills. They've just been marketed that way.

GolDDranks 9y ago

Indeed, the virtue of the Turing test is that it's a very general framework – the freedom of the framework allows us to test the AI using any communicational method at hand. One of such is Winograd Schemas [1] – a test of commonsensical reasoning, in which the current conversational AIs spectacularly fail at. [1] https://cs.nyu.edu/faculty/davise/papers/WinogradSchemas/WS.html https://cs.nyu.edu/faculty/davise/papers/WinogradSchemas/WS....

GolDDranks 9y ago
Btw. here is a sketch of an embedding like that: [Session starts] <Challenger> Hi. <Judge> Hi. Now, listen to me. My job here is to judge whether you are a human or an AI. I also know that your job is to try and persuade me to believe you are a human, no matter which one you really are. I happen to have a few questions that are easy for humans and hard for AIs. So if you are a human, answering them shouldn't be a problem to you, but if you avoid answering, I'm going to judge that you are an AI, no excuses. Here is the first question: ... [A pattern of Winograd schema challenges ensues.] So as we can see, the Turing test doesn't have to be any random chitchat nor it doesn't have to pretend to be "normal human-to-human-like discussion", as the Judge has the ability to participate and steer the conversation and zero in on the areas that are most likely to expose the AI. However, if the human participants don't have any penalty from being labeled as AI, this doesn't obviously work, as they don't have any pressure to prove anything. So this kind of an embedding wouldn't work with "imitation game" -like rules, but it would work with "prove your human-ness" -like rules.
- Retric 9y ago
  If you classify say 50% of people as AI's and 50% of AI's as people then those AI's passed the Turing test. So, you are limited to questions that most people will answer correctly. Further, if you find some unusual question that works today someone can just add it to the program for next time.
  - GolDDranks 9y ago
    Did you check the link about the Winograd schema challenge? The questions test common sense reasoning, and are very easy for humans to answer. An example: The trophy doesn't fit into the brown suitcase because it's too large. What is too large? A: The trophy B: The suitcase
    - Retric 9y ago
      Coding a specific solution to questions in exactly that format is not inherently difficult. If you keep writing specific solutions to this class of questions it's eventually had to come up with a simple question that's still unknown. Further you can't get around that by making ever more complex questions because humans will eventually start messing them up.
  - edmccard 9y ago
    >If you classify say 50% of people as AI's and 50% of AI's as people then those AI's passed the Turing test. And if your version of the test is "heads it's an AI, tails it's human" then any AI's that are classified as human will have "passed the Turing test."
    - Retric 9y ago
      I don't think you understood. The original test specifically had exactly one human and one AI. So, if the judge is forced to do a coin flip that really is success. If the judge does a coin flip because they are lazy then that's not a Turning test.
- red75prime 9y ago

marzell 9y ago

I feel we could have reached that point by now, for instance is there was a Turing Race a la the Space Race. The driving factor though is we don't specifically need that from technology. Even when we get to that point, there's tons of reasons we will often want to easily distinguish whether we are communicating with someone vs something. Being distinguishable doesn't mean an AI is any less effective or easy to communicate with. And who knows, perhaps this tech is already in the wild - by definition wouldn't it be fooling us?

Retric 9y ago

Computers have passed limited versions of the Turing test, as in one person was mistakenly classified as AI and an AI classified as a person. Now, I am not saying we could have an expert chat with one for an hour on any topic and still be fooled, but people have already chatted with an AI without noticing it was an AI an not a person. AGI is a different problem. But, faking AGI to the point someone can't tell is more of a philosophical question than a failure of the Turing test.

olavk 9y ago
people have already chatted with an AI without noticing it was an AI an not a person I'm sure this have worked in some limited context where the human was not actively trying to figure out if it was a human or AI. I mean, I have sometimes at a distance mistaken a mannequin for a human - that does not mean mannequins are now indistinguishable from humans. In the Turing tests the judge is active trying to discern if they are talking to a (adult, literate, mentally sound) human or an AI. AFAIK no AI have ever passed this. (Of course it is easy for a human to impersonate a bad AI by answering erratically, but that doesn't really prove anything.)
- JoeAltmaier 9y ago
  I'm not sure that masquerading as a normal human being is any kind of a bar for an AI to pass. How about, passing for an expert in a field e.g. medicine, investing, navigation? I'd much rather see my AI certified that way.
  - olavk 9y ago
    Such a specialized AI would probably be more useful in practice, but that is not really what the Turing test is concerned with. The Turing test is a behaviorist approach to deciding the question if a computer "can think". In any case, it think it is much easier to create an AI specialized in a particular domain rather than a general "human-like" AI.
    - JoeAltmaier 9y ago