6 ms·

(1) As it is with humans. It's not possible to attach a debugger to a human brain and determine the nature of understanding; you have to ask the human questions

by krona 4y ago

(1) As it is with humans. It's not possible to attach a debugger to a human brain and determine the nature of understanding; you have to ask the human questions about that understanding and interpret the response yourself. (2) This is a strange one to me. It's a bit like saying an essential quality of a human is to experience the world as a human, and therefore nothing except a human can be human. (3) If only all humans met this arbitrary standard of secular scientific rationalism. (4) I'd say 5% of humanity fail to meet this criteria.

mjburgess 4y ago
(1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) The reason we think people have the capacity to interpret sentences is their context-appropriate sensory-motor actions, reasoning, emotion, etc. When I say, "quick! a fire!" people: panic, run, get organised, etc. There are a very very large number of highly complex environmental, interpersonal, sensory-motor, emotional, etc. etc. interactions which follow. ChatGPT not only does not display any, but cannot. This isn't up for debate, the system has no capacity to understand text. If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Worse, as a matter of fact, we know its possible to generate output as-if it did understand. So we (1) have known a mechanism whereby it can fake it; and (2) know that this mechanism is the one its using. (2) All thinking possess intentionality which simply means its about something, and the reason we ask, say, "should i call the police?!" is because we are situated in an environment, which causes us to form mental states about that environment. Any system which emits, "should I call the police?" without having a backing representational state of, eg., the police, care, concern, priority, the environment which requires police, etc. *Does Not Mean* this question. It cannot. (3) The capacity to speak truthfully does not mean you speak truthfully. That capacity is simply that you can form representational states which you can introspect as being accurate or inaccurate. ChatGPT lacks that capacity. It does not form representational (intentional) states when it generates text; each sentence is not caused by such a state. (4) The capacity i named was "caring", pretty much all people care, it's necessary for goal-directed action.
- krona 4y ago
  ChatGPT has state and representation, clearly. But that's besides the point. It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve. All of this is a bit like saying a car isn't a horse because it doesn't have hooves. Yes, but so what?
  - mjburgess 4y ago
    The question of whether it has a representational model of the terms it's using is open at the moment. I'm strongly inclined to deny it, but the argument on this is at least quite complex and subject to premises which are less certain than the following: It does not form a representational state prior to speaking which is the reason it speaks. In other-words, it does not say, "I like ice-cream" because it likes ice-cream. That's sufficient to invalidate the idea that it has any relevant capacities here. We even know, for example, that it randomly chooses unlikely words to make its output seem more reasonable: this mechanism alone invalidates it having a representational model. --- And on the former point: if we look at pretty much all plausible theories of meaning, the meaning of terms cannot be anything like what ChatGPT has access to. I think what ChatGPT has is something like "a statistically approximate model of word connotation". Whether a statistical approximation of part of meaning counts as "being meaningful" is a matter of debate. (I think it clearly doesnt.)
    - yunwal 4y ago
      > It does not form a representational state prior to speaking which is the reason it speaks. The whole thing is a representational state, and the fact that it's identical at the start of every conversation is a design choice, not inherent to the way the model learns. > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. You also don't say "I like ice-cream" because you like ice cream. You say it because through years of evolution you've been taught to crave fat and sugar and other things in ice cream, which fuels your body, and helps you live and reproduce. The fact that this AI is optimizing for something else doesn't really prove anything. I also don't think that we have anything more than a "statistically approximate model" of anything outside of our own thoughts. The theory of gravity exists because we've seen objects fall from heights a million times, we're told by others we speak to that they experience the same thing. From this we can say that with 99.9999...% (enough 9s to be indistinguishable from 100) probability, the theory of gravity is true.
gooseus 4y ago
If anything terrifies me about our current moment in AI, it's this impulse to dehumanize actual living people as a way of defending AI from entirely valid criticism. Sure, people suck and lots of them are immoral and stupid... but they're still human beings who could have been otherwise (counterfactual thinking and all that) given a different "training set" (upbringing). Unless you're implying that some percentage of human beings are incapable (like ChatGPT) of acquiring a morality or rational scientific understanding, no matter their training?
- mjburgess 4y ago
  Yes, likewise. I find it extremely concerning that people are ready to believe, about themselves that the reason they say, "I love you!" is because they're just correlating patterns in their memories of every time someone mentioned those words.
  - shrimp_emoji 4y ago
    Mental grown-ups don't believe in souls. Or sometimes other religious concepts like "consciousness". ;D (That one takes more thinking through, though, imo.)
  - yunwal 4y ago
    Why? Are we not allowed to find meaning, beauty, in patterns? The fact that two people can share experiences that bond them together, allow them to deeply trust each other, share a sense of humor, etc, isn't not beautiful just because there's some extremely complex statistical calculations going on under the surface that we ourselves can't explain. It's like saying it's concerning that human beings are ready to believe they're made up of combinations of atoms. What would be an appropriate explanation for the concept of love? Or should we just never go beyond magic?
    - agalunar 4y ago
      The point is that love isn't just a linguistic phenomenon – it's some other internal phenomenon or experience that happens to cause linguistic expression (among other forms of expression or behavior modulation.) Love is something we feel; words are evidence of it. We don't define love as a pattern that shows up in a series of conversations. There are many other ways of generating those same words. Nobody's trying to reject materialism here. I myself happen to believe that love is nothing more than (or at least, can entirely be explained by) electrochemical processes in my brain. And I agree that that doesn't make love any less magical ^_^ I think by "correlating patterns", the person you're replying to meant "doing just enough processing to produce the salient performances". But it's possible to have additional "inner mental life" beyond the minimum that's necessary to give rise to an outward behavior. (In this case, the outward behavior is speech.) A single interface can have many different implementations, as it were.
genman 4y ago
(4) You are really generous.

by krona 4y ago

mjburgess 4y ago

(1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) The reason we think people have the capacity to interpret sentences is their context-appropriate sensory-motor actions, reasoning, emotion, etc. When I say, "quick! a fire!" people: panic, run, get organised, etc. There are a very very large number of highly complex environmental, interpersonal, sensory-motor, emotional, etc. etc. interactions which follow. ChatGPT not only does not display any, but cannot. This isn't up for debate, the system has no capacity to understand text. If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Worse, as a matter of fact, we know its possible to generate output as-if it did understand. So we (1) have known a mechanism whereby it can fake it; and (2) know that this mechanism is the one its using. (2) All thinking possess intentionality which simply means its about something, and the reason we ask, say, "should i call the police?!" is because we are situated in an environment, which causes us to form mental states about that environment. Any system which emits, "should I call the police?" without having a backing representational state of, eg., the police, care, concern, priority, the environment which requires police, etc. *Does Not Mean* this question. It cannot. (3) The capacity to speak truthfully does not mean you speak truthfully. That capacity is simply that you can form representational states which you can introspect as being accurate or inaccurate. ChatGPT lacks that capacity. It does not form representational (intentional) states when it generates text; each sentence is not caused by such a state. (4) The capacity i named was "caring", pretty much all people care, it's necessary for goal-directed action.

krona 4y ago
ChatGPT has state and representation, clearly. But that's besides the point. It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve. All of this is a bit like saying a car isn't a horse because it doesn't have hooves. Yes, but so what?
- mjburgess 4y ago
  The question of whether it has a representational model of the terms it's using is open at the moment. I'm strongly inclined to deny it, but the argument on this is at least quite complex and subject to premises which are less certain than the following: It does not form a representational state prior to speaking which is the reason it speaks. In other-words, it does not say, "I like ice-cream" because it likes ice-cream. That's sufficient to invalidate the idea that it has any relevant capacities here. We even know, for example, that it randomly chooses unlikely words to make its output seem more reasonable: this mechanism alone invalidates it having a representational model. --- And on the former point: if we look at pretty much all plausible theories of meaning, the meaning of terms cannot be anything like what ChatGPT has access to. I think what ChatGPT has is something like "a statistically approximate model of word connotation". Whether a statistical approximation of part of meaning counts as "being meaningful" is a matter of debate. (I think it clearly doesnt.)
  - yunwal 4y ago
    > It does not form a representational state prior to speaking which is the reason it speaks. The whole thing is a representational state, and the fact that it's identical at the start of every conversation is a design choice, not inherent to the way the model learns. > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. You also don't say "I like ice-cream" because you like ice cream. You say it because through years of evolution you've been taught to crave fat and sugar and other things in ice cream, which fuels your body, and helps you live and reproduce. The fact that this AI is optimizing for something else doesn't really prove anything. I also don't think that we have anything more than a "statistically approximate model" of anything outside of our own thoughts. The theory of gravity exists because we've seen objects fall from heights a million times, we're told by others we speak to that they experience the same thing. From this we can say that with 99.9999...% (enough 9s to be indistinguishable from 100) probability, the theory of gravity is true.

gooseus 4y ago

If anything terrifies me about our current moment in AI, it's this impulse to dehumanize actual living people as a way of defending AI from entirely valid criticism. Sure, people suck and lots of them are immoral and stupid... but they're still human beings who could have been otherwise (counterfactual thinking and all that) given a different "training set" (upbringing). Unless you're implying that some percentage of human beings are incapable (like ChatGPT) of acquiring a morality or rational scientific understanding, no matter their training?

mjburgess 4y ago
Yes, likewise. I find it extremely concerning that people are ready to believe, about themselves that the reason they say, "I love you!" is because they're just correlating patterns in their memories of every time someone mentioned those words.
- shrimp_emoji 4y ago
  Mental grown-ups don't believe in souls. Or sometimes other religious concepts like "consciousness". ;D (That one takes more thinking through, though, imo.)
- yunwal 4y ago
  Why? Are we not allowed to find meaning, beauty, in patterns? The fact that two people can share experiences that bond them together, allow them to deeply trust each other, share a sense of humor, etc, isn't not beautiful just because there's some extremely complex statistical calculations going on under the surface that we ourselves can't explain. It's like saying it's concerning that human beings are ready to believe they're made up of combinations of atoms. What would be an appropriate explanation for the concept of love? Or should we just never go beyond magic?
  - agalunar 4y ago
    The point is that love isn't just a linguistic phenomenon – it's some other internal phenomenon or experience that happens to cause linguistic expression (among other forms of expression or behavior modulation.) Love is something we feel; words are evidence of it. We don't define love as a pattern that shows up in a series of conversations. There are many other ways of generating those same words. Nobody's trying to reject materialism here. I myself happen to believe that love is nothing more than (or at least, can entirely be explained by) electrochemical processes in my brain. And I agree that that doesn't make love any less magical ^_^ I think by "correlating patterns", the person you're replying to meant "doing just enough processing to produce the salient performances". But it's possible to have additional "inner mental life" beyond the minimum that's necessary to give rise to an outward behavior. (In this case, the outward behavior is speech.) A single interface can have many different implementations, as it were.

genman 4y ago

(4) You are really generous.