4 ms·

You say: “The Turing test is a meaningless benchmark” and then in the next sentence you say: “there’s not anything today that would come remotely close to passi

by bryan0 4y ago

You say: “The Turing test is a meaningless benchmark” and then in the next sentence you say: “there’s not anything today that would come remotely close to passing the Turing test” So which is it? I think you’re confused because there’s a difference between fooling a random person and passing a Turing test as described by Kurzweil [0] The magician analogy is kind of like James Randi’s Paranormal test [1]. Again a proper test should determine whether the magic is real or a trick. [0] https://www.kurzweilai.net/a-wager-on-the-turing-test-the-rules https://www.kurzweilai.net/a-wager-on-the-turing-test-the-ru... [1] https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranormal_Challenge https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranorma...

HarHarVeryFunny 4y ago
> So which is it? Given any well-defined (Kurzweil/whatever) variant of the test, we can obviously test any entity and see if it passes. However, my assertion is that passing the test is "meaningless" in sense of saying anything profound about the entity that passed. So, there is no either/or consideration here - we can discuss an entity passing the test (benchmark) regardless of what importance we attach to the result. My point was simply that fooling people into believing something isn't the same as that something actually being true. Additionally, as far as any "intelligence" (and/or "consciousness") test goes, IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure ("intelligent in the opinion of the test supervisor"). I'm OK with the notion of using a black box test to gauge intelligence (since I regard it as behavioral quality), but we have to be careful when comparing to a human (as the Turing test does) since we could allow the bar to be arbitrarily low by not demanding the performance of a fully intelligent and unhampered human (can ChatGPT convince someone it's an idiot with OCD stuck in a box looking up answers in a book it doesn't understand ... maybe. can it convince me it's an averagely intelligent student able to perform experiments and report on the outcome - not even remotely close).
- bryan0 4y ago
  Yeah I think you make good points. > IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure. This what I think is the beauty of the Turing test because trying to agree on a definition for consciousness, sentience, understanding, etc.. is futile. The Turing test says who cares about those definitions, instead look at whether a computer in general conversation is indistinguishable from a human. Because that’s really what matters.
  - HarHarVeryFunny 4y ago
    Yes, I suppose that's what the Turing test is meant to do - test for intelligence without having to define it, and the implicit definition of ("if it acts intelligent, it is intelligent") is reasonable enough, but I'm really not sure what use it has! It was an interesting thought experiment in 1950 at the beginning of the computer age, but now that we're getting close to actually being able to build intelligent systems, what's the point ?! Artificial intelligence will exist on a spectrum from well-below to well-beyond human, and it'd be useful to have some way to quantify that rather than just be able to say "this one's right around human level!". We really need an "ImageNet for AI" competition that can measure intelligence. I think we can certainly define intelligence (and consciousness too for that matter), even if not everyone is going to agree. Given human nature there'll still be people denying machine intelligence well after they're being managed by machines ! My rough take definition of intelligence is "ability to predict", IOW degree to which an entity can use prior experience to anticipate and plan for the future, with plans unfolding as predicted. Given that the human cortex appears to be literally a prediction machine (learning by prediction failure), this definition seems well matched to the notion of "if it acts human, its intelligent", but it's specific enough that I think one could devise tests requiring different levels of learning (from experience) and prediction to quantify it.
    - bryan0 4y ago
      > now that we're getting close to actually being able to build intelligent systems, what's the point ?! What’s the point of going to the moon? It would be a great human achievement and life will never be the same. It also doesn’t need to be binary while machines are attempting to pass it. You still would be able to judge how close computers are getting as the questions in a Turing test get more and more nuanced and sophisticated. That being said we should have more objective tests like you described. No reason we wont do both.

by bryan0 4y ago

HarHarVeryFunny 4y ago

> So which is it? Given any well-defined (Kurzweil/whatever) variant of the test, we can obviously test any entity and see if it passes. However, my assertion is that passing the test is "meaningless" in sense of saying anything profound about the entity that passed. So, there is no either/or consideration here - we can discuss an entity passing the test (benchmark) regardless of what importance we attach to the result. My point was simply that fooling people into believing something isn't the same as that something actually being true. Additionally, as far as any "intelligence" (and/or "consciousness") test goes, IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure ("intelligent in the opinion of the test supervisor"). I'm OK with the notion of using a black box test to gauge intelligence (since I regard it as behavioral quality), but we have to be careful when comparing to a human (as the Turing test does) since we could allow the bar to be arbitrarily low by not demanding the performance of a fully intelligent and unhampered human (can ChatGPT convince someone it's an idiot with OCD stuck in a box looking up answers in a book it doesn't understand ... maybe. can it convince me it's an averagely intelligent student able to perform experiments and report on the outcome - not even remotely close).

bryan0 4y ago
Yeah I think you make good points. > IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure. This what I think is the beauty of the Turing test because trying to agree on a definition for consciousness, sentience, understanding, etc.. is futile. The Turing test says who cares about those definitions, instead look at whether a computer in general conversation is indistinguishable from a human. Because that’s really what matters.
- HarHarVeryFunny 4y ago
  Yes, I suppose that's what the Turing test is meant to do - test for intelligence without having to define it, and the implicit definition of ("if it acts intelligent, it is intelligent") is reasonable enough, but I'm really not sure what use it has! It was an interesting thought experiment in 1950 at the beginning of the computer age, but now that we're getting close to actually being able to build intelligent systems, what's the point ?! Artificial intelligence will exist on a spectrum from well-below to well-beyond human, and it'd be useful to have some way to quantify that rather than just be able to say "this one's right around human level!". We really need an "ImageNet for AI" competition that can measure intelligence. I think we can certainly define intelligence (and consciousness too for that matter), even if not everyone is going to agree. Given human nature there'll still be people denying machine intelligence well after they're being managed by machines ! My rough take definition of intelligence is "ability to predict", IOW degree to which an entity can use prior experience to anticipate and plan for the future, with plans unfolding as predicted. Given that the human cortex appears to be literally a prediction machine (learning by prediction failure), this definition seems well matched to the notion of "if it acts human, its intelligent", but it's specific enough that I think one could devise tests requiring different levels of learning (from experience) and prediction to quantify it.
  - bryan0 4y ago
    > now that we're getting close to actually being able to build intelligent systems, what's the point ?! What’s the point of going to the moon? It would be a great human achievement and life will never be the same. It also doesn’t need to be binary while machines are attempting to pass it. You still would be able to judge how close computers are getting as the questions in a Turing test get more and more nuanced and sophisticated. That being said we should have more objective tests like you described. No reason we wont do both.