4 ms·

This article made more sense pre-GPT. Now I’m not so sure. I think if a transformer based architecture was able to live update its weights and was trained on th

by bryan0 4y ago

This article made more sense pre-GPT. Now I’m not so sure. I think if a transformer based architecture was able to live update its weights and was trained on the proper abstraction of data, then it could possibly pass the Turing test. Edit: actually I take it back. This article would not have made any sense pre-GPT either.

HarHarVeryFunny 4y ago
The Turing test is a meaningless benchmark. So what if you can fool people? It's like saying if a magician can fool people then they really have supernatural powers. If you believe the magician really has powers to make stuff materialize out of thin air, then you'd be disappointed they couldn't just materialize you a million bucks, and if you believe some chatbot that passed the Turing test really is intelligent, then you'd be disappointed it couldn't help you make a million bucks in the stock market, or do anything that actually required real intelligence vs the faked appearance of intelligence. Anyway, there's not even anything today that would come remotely close to passing the Turing test. I guess ChatGPT might fool some people into thinking it's an idiot who was locked in a box with a printout of WikiPedia last year and has no access to the outside world, but not that it's a normal person with access to, and able to interact with, the outside world.
- bryan0 4y ago
  You say: “The Turing test is a meaningless benchmark” and then in the next sentence you say: “there’s not anything today that would come remotely close to passing the Turing test” So which is it? I think you’re confused because there’s a difference between fooling a random person and passing a Turing test as described by Kurzweil [0] The magician analogy is kind of like James Randi’s Paranormal test [1]. Again a proper test should determine whether the magic is real or a trick. [0] https://www.kurzweilai.net/a-wager-on-the-turing-test-the-rules https://www.kurzweilai.net/a-wager-on-the-turing-test-the-ru... [1] https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranormal_Challenge https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranorma...
  - HarHarVeryFunny 4y ago
    > So which is it? Given any well-defined (Kurzweil/whatever) variant of the test, we can obviously test any entity and see if it passes. However, my assertion is that passing the test is "meaningless" in sense of saying anything profound about the entity that passed. So, there is no either/or consideration here - we can discuss an entity passing the test (benchmark) regardless of what importance we attach to the result. My point was simply that fooling people into believing something isn't the same as that something actually being true. Additionally, as far as any "intelligence" (and/or "consciousness") test goes, IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure ("intelligent in the opinion of the test supervisor"). I'm OK with the notion of using a black box test to gauge intelligence (since I regard it as behavioral quality), but we have to be careful when comparing to a human (as the Turing test does) since we could allow the bar to be arbitrarily low by not demanding the performance of a fully intelligent and unhampered human (can ChatGPT convince someone it's an idiot with OCD stuck in a box looking up answers in a book it doesn't understand ... maybe. can it convince me it's an averagely intelligent student able to perform experiments and report on the outcome - not even remotely close).
    - bryan0 4y ago
      Yeah I think you make good points. > IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure. This what I think is the beauty of the Turing test because trying to agree on a definition for consciousness, sentience, understanding, etc.. is futile. The Turing test says who cares about those definitions, instead look at whether a computer in general conversation is indistinguishable from a human. Because that’s really what matters.
      - HarHarVeryFunny 4y ago
        Yes, I suppose that's what the Turing test is meant to do - test for intelligence without having to define it, and the implicit definition of ("if it acts intelligent, it is intelligent") is reasonable enough, but I'm really not sure what use it has! It was an interesting thought experiment in 1950 at the beginning of the computer age, but now that we're getting close to actually being able to build intelligent systems, what's the point ?! Artificial intelligence will exist on a spectrum from well-below to well-beyond human, and it'd be useful to have some way to quantify that rather than just be able to say "this one's right around human level!". We really need an "ImageNet for AI" competition that can measure intelligence. I think we can certainly define intelligence (and consciousness too for that matter), even if not everyone is going to agree. Given human nature there'll still be people denying machine intelligence well after they're being managed by machines ! My rough take definition of intelligence is "ability to predict", IOW degree to which an entity can use prior experience to anticipate and plan for the future, with plans unfolding as predicted. Given that the human cortex appears to be literally a prediction machine (learning by prediction failure), this definition seems well matched to the notion of "if it acts human, its intelligent", but it's specific enough that I think one could devise tests requiring different levels of learning (from experience) and prediction to quantify it.

by bryan0 4y ago

HarHarVeryFunny 4y ago

The Turing test is a meaningless benchmark. So what if you can fool people? It's like saying if a magician can fool people then they really have supernatural powers. If you believe the magician really has powers to make stuff materialize out of thin air, then you'd be disappointed they couldn't just materialize you a million bucks, and if you believe some chatbot that passed the Turing test really is intelligent, then you'd be disappointed it couldn't help you make a million bucks in the stock market, or do anything that actually required real intelligence vs the faked appearance of intelligence. Anyway, there's not even anything today that would come remotely close to passing the Turing test. I guess ChatGPT might fool some people into thinking it's an idiot who was locked in a box with a printout of WikiPedia last year and has no access to the outside world, but not that it's a normal person with access to, and able to interact with, the outside world.

bryan0 4y ago
You say: “The Turing test is a meaningless benchmark” and then in the next sentence you say: “there’s not anything today that would come remotely close to passing the Turing test” So which is it? I think you’re confused because there’s a difference between fooling a random person and passing a Turing test as described by Kurzweil [0] The magician analogy is kind of like James Randi’s Paranormal test [1]. Again a proper test should determine whether the magic is real or a trick. [0] https://www.kurzweilai.net/a-wager-on-the-turing-test-the-rules https://www.kurzweilai.net/a-wager-on-the-turing-test-the-ru... [1] https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranormal_Challenge https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranorma...
- HarHarVeryFunny 4y ago
  > So which is it? Given any well-defined (Kurzweil/whatever) variant of the test, we can obviously test any entity and see if it passes. However, my assertion is that passing the test is "meaningless" in sense of saying anything profound about the entity that passed. So, there is no either/or consideration here - we can discuss an entity passing the test (benchmark) regardless of what importance we attach to the result. My point was simply that fooling people into believing something isn't the same as that something actually being true. Additionally, as far as any "intelligence" (and/or "consciousness") test goes, IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure ("intelligent in the opinion of the test supervisor"). I'm OK with the notion of using a black box test to gauge intelligence (since I regard it as behavioral quality), but we have to be careful when comparing to a human (as the Turing test does) since we could allow the bar to be arbitrarily low by not demanding the performance of a fully intelligent and unhampered human (can ChatGPT convince someone it's an idiot with OCD stuck in a box looking up answers in a book it doesn't understand ... maybe. can it convince me it's an averagely intelligent student able to perform experiments and report on the outcome - not even remotely close).
  - bryan0 4y ago
    Yeah I think you make good points. > IMO it's even more meaningless if we attempt to test for these before defining precisely what we mean by these terms, since otherwise passing is a subjective measure. This what I think is the beauty of the Turing test because trying to agree on a definition for consciousness, sentience, understanding, etc.. is futile. The Turing test says who cares about those definitions, instead look at whether a computer in general conversation is indistinguishable from a human. Because that’s really what matters.
    - HarHarVeryFunny 4y ago
      Yes, I suppose that's what the Turing test is meant to do - test for intelligence without having to define it, and the implicit definition of ("if it acts intelligent, it is intelligent") is reasonable enough, but I'm really not sure what use it has! It was an interesting thought experiment in 1950 at the beginning of the computer age, but now that we're getting close to actually being able to build intelligent systems, what's the point ?! Artificial intelligence will exist on a spectrum from well-below to well-beyond human, and it'd be useful to have some way to quantify that rather than just be able to say "this one's right around human level!". We really need an "ImageNet for AI" competition that can measure intelligence. I think we can certainly define intelligence (and consciousness too for that matter), even if not everyone is going to agree. Given human nature there'll still be people denying machine intelligence well after they're being managed by machines ! My rough take definition of intelligence is "ability to predict", IOW degree to which an entity can use prior experience to anticipate and plan for the future, with plans unfolding as predicted. Given that the human cortex appears to be literally a prediction machine (learning by prediction failure), this definition seems well matched to the notion of "if it acts human, its intelligent", but it's specific enough that I think one could devise tests requiring different levels of learning (from experience) and prediction to quantify it.