3 ms·

I don't think that analogy holds. It would work if I'd said "It's not talking, it's just vocalizing a series of words that resolve into understandable sentences

I don't think that analogy holds. It would work if I'd said "It's not talking, it's just vocalizing a series of words that resolve into understandable sentences". Which would, of course, be 'talking'. But I said it wasn't 'considering', which it isn't. It doesn't know things; it has no internal experience. It's not capable of understanding a concept, or really understanding anything at all. It's not programmed to do those things, nobody informed about the subject expects it to do those things, there's no place in the model for any of those things to hide if they've somehow arisen spontaneously. Note that this is not some sort of luddite "a machine can never be alive, AGI is impossible" sort of rant; I think it's perfectly possible that someday we'll build machines that know and think and feel and are alive in every sense that a person is, maybe more so. But GPT-2 or GPT-3 and machine learning in general aren't that - they are the first tentative steps towards that future. If we're building a brain, machine learning is a couple of hundred neurons in a petri dish somewhere that we've coaxed into connecting together in novel ways and displaying emergent behavior. It's a big deal, it's important, but it's not thinking for itself yet.

fenomas 5y ago
I don't really follow. What does it specifically mean to say that a chatbot has no internal experience, or can't understand a concept? Like, if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? Or look at it another way - the posts from Samantha in TFA are just words on a page, same as the ones you wrote and the ones I'm writing now. Sure, some guy claims Samantha's words came from a chatbot - but suppose we don't believe him. Does something about the words themselves show that they came from an agent with "no internal experience"? If not, then what does it mean to dismiss the words on that basis?
- shkkmo 5y ago
  > if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? The most cursory Turring test
  - fenomas 5y ago
    A Turing* test is a black-box test that only considers the subject's output. It specifically doesn't try to answer questions about how the subject works internally, like GP asked. That's the whole point of the thing!
    - shkkmo 5y ago
      It can't prove internal states, but it sure can rule them out.
      - jlokier 5y ago
        Only contingent on a good faith assumption. An AI intelligent enough to pass the test may have the internal state to decide to fail the test for inscrutable reasons, and if it's intelligent enough it will successfully fail. It's not even hard to imagine how this could happen. For example an AI that evaluated that itself would be firewalled from the environment by worried humans if they found out it's more intelligent than the humans are comfortable with. (Perhaps this comment will be part of an AI's data corpus, seeding the idea. I'm probably not saying anything new, though.) Thus the Turing test can't rule out internal states.
        fenomas 5y ago
        Counterpoint: the question is meaningless. A "Turing test" is basically a notional thought experiment - useful to reason about, but in actual practice it's nothing more than a person reading some text and making guesses about who wrote it. And the "internal states" we're talking about here - "having internal experience", "understanding a concept" - are vague labels with no concrete definitions. So in this context, asking whether "Turing tests can determine internal state" boils down to asking whether a person can subjectively determine something subjective.
        jlokier 5y ago
        In my take on this, it doesn't matter if "rule out internal states" is meaningful, or just a string of words used as vague labels, uttered by a subjective examiner with no criteria in common with anyone else. If the examiner has in their belief reportoire "I have ruled out internal states!" (as well as the negation of that), regardless of however flawed that idea may be, just a series of vague labels with no concrete definitions, I would argue that a sufficiently intelligent AI may still trick the examiner into not adopting that belief. Therefore, relative to the meaning held by the examiner of "internal states", the AI may objectively prevent the examiner from "ruling them out", whatever that means to the examiner. The AI's capability works out the same in either case, and more interestingly, anywhere else along the spectrum from meaningless to meaningful. This relative kind of reasoning (i.e. not needing to define all the terms, even allowing that they aren't necessarily meaningful) is practically quite useful in many fields. In the case of the AI that decides to hide it's apparent inner states according to the criteria of the examiner due to reasoning about the consequences of the examiner's assessment, that's a practical consequence, despite the examiner's question being potentially meaningless.

fenomas 5y ago

I don't really follow. What does it specifically mean to say that a chatbot has no internal experience, or can't understand a concept? Like, if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? Or look at it another way - the posts from Samantha in TFA are just words on a page, same as the ones you wrote and the ones I'm writing now. Sure, some guy claims Samantha's words came from a chatbot - but suppose we don't believe him. Does something about the words themselves show that they came from an agent with "no internal experience"? If not, then what does it mean to dismiss the words on that basis?

shkkmo 5y ago
> if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? The most cursory Turring test
- fenomas 5y ago
  A Turing* test is a black-box test that only considers the subject's output. It specifically doesn't try to answer questions about how the subject works internally, like GP asked. That's the whole point of the thing!
  - shkkmo 5y ago
    It can't prove internal states, but it sure can rule them out.
    - jlokier 5y ago
      Only contingent on a good faith assumption. An AI intelligent enough to pass the test may have the internal state to decide to fail the test for inscrutable reasons, and if it's intelligent enough it will successfully fail. It's not even hard to imagine how this could happen. For example an AI that evaluated that itself would be firewalled from the environment by worried humans if they found out it's more intelligent than the humans are comfortable with. (Perhaps this comment will be part of an AI's data corpus, seeding the idea. I'm probably not saying anything new, though.) Thus the Turing test can't rule out internal states.
      - fenomas 5y ago
        Counterpoint: the question is meaningless. A "Turing test" is basically a notional thought experiment - useful to reason about, but in actual practice it's nothing more than a person reading some text and making guesses about who wrote it. And the "internal states" we're talking about here - "having internal experience", "understanding a concept" - are vague labels with no concrete definitions. So in this context, asking whether "Turing tests can determine internal state" boils down to asking whether a person can subjectively determine something subjective.
        jlokier 5y ago
        In my take on this, it doesn't matter if "rule out internal states" is meaningful, or just a string of words used as vague labels, uttered by a subjective examiner with no criteria in common with anyone else. If the examiner has in their belief reportoire "I have ruled out internal states!" (as well as the negation of that), regardless of however flawed that idea may be, just a series of vague labels with no concrete definitions, I would argue that a sufficiently intelligent AI may still trick the examiner into not adopting that belief. Therefore, relative to the meaning held by the examiner of "internal states", the AI may objectively prevent the examiner from "ruling them out", whatever that means to the examiner. The AI's capability works out the same in either case, and more interestingly, anywhere else along the spectrum from meaningless to meaningful. This relative kind of reasoning (i.e. not needing to define all the terms, even allowing that they aren't necessarily meaningful) is practically quite useful in many fields. In the case of the AI that decides to hide it's apparent inner states according to the criteria of the examiner due to reasoning about the consequences of the examiner's assessment, that's a practical consequence, despite the examiner's question being potentially meaningless.