5 ms·

(1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) The reason we th

(1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) The reason we think people have the capacity to interpret sentences is their context-appropriate sensory-motor actions, reasoning, emotion, etc. When I say, "quick! a fire!" people: panic, run, get organised, etc. There are a very very large number of highly complex environmental, interpersonal, sensory-motor, emotional, etc. etc. interactions which follow. ChatGPT not only does not display any, but cannot. This isn't up for debate, the system has no capacity to understand text. If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Worse, as a matter of fact, we know its possible to generate output as-if it did understand. So we (1) have known a mechanism whereby it can fake it; and (2) know that this mechanism is the one its using. (2) All thinking possess intentionality which simply means its about something, and the reason we ask, say, "should i call the police?!" is because we are situated in an environment, which causes us to form mental states about that environment. Any system which emits, "should I call the police?" without having a backing representational state of, eg., the police, care, concern, priority, the environment which requires police, etc. *Does Not Mean* this question. It cannot. (3) The capacity to speak truthfully does not mean you speak truthfully. That capacity is simply that you can form representational states which you can introspect as being accurate or inaccurate. ChatGPT lacks that capacity. It does not form representational (intentional) states when it generates text; each sentence is not caused by such a state. (4) The capacity i named was "caring", pretty much all people care, it's necessary for goal-directed action.

krona 4y ago
ChatGPT has state and representation, clearly. But that's besides the point. It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve. All of this is a bit like saying a car isn't a horse because it doesn't have hooves. Yes, but so what?
- mjburgess 4y ago
  The question of whether it has a representational model of the terms it's using is open at the moment. I'm strongly inclined to deny it, but the argument on this is at least quite complex and subject to premises which are less certain than the following: It does not form a representational state prior to speaking which is the reason it speaks. In other-words, it does not say, "I like ice-cream" because it likes ice-cream. That's sufficient to invalidate the idea that it has any relevant capacities here. We even know, for example, that it randomly chooses unlikely words to make its output seem more reasonable: this mechanism alone invalidates it having a representational model. --- And on the former point: if we look at pretty much all plausible theories of meaning, the meaning of terms cannot be anything like what ChatGPT has access to. I think what ChatGPT has is something like "a statistically approximate model of word connotation". Whether a statistical approximation of part of meaning counts as "being meaningful" is a matter of debate. (I think it clearly doesnt.)
  - yunwal 4y ago
    > It does not form a representational state prior to speaking which is the reason it speaks. The whole thing is a representational state, and the fact that it's identical at the start of every conversation is a design choice, not inherent to the way the model learns. > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. You also don't say "I like ice-cream" because you like ice cream. You say it because through years of evolution you've been taught to crave fat and sugar and other things in ice cream, which fuels your body, and helps you live and reproduce. The fact that this AI is optimizing for something else doesn't really prove anything. I also don't think that we have anything more than a "statistically approximate model" of anything outside of our own thoughts. The theory of gravity exists because we've seen objects fall from heights a million times, we're told by others we speak to that they experience the same thing. From this we can say that with 99.9999...% (enough 9s to be indistinguishable from 100) probability, the theory of gravity is true.
    - simple-thoughts 4y ago
      I would amend that there is valid reasoning other than empirical analysis, but also that chatgpt is capable of generating text demonstrating those types of reasoning.
- ahahahahah 4y ago
  > It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve That's wrong. It's been demonstrated to produce responses to question such that if a 9 year old child would produce those same responses we would assume that that child had a typical 9 year-olds theory of mind. Not even for the child does "passing" such a test demonstrate having such theory of mind, though given our understanding of how humans develop it's a pretty good signal. I don't know how someone could understand that and still think that chatgpt has demonstrated theory of mind. It is, in fact, the same for a lot of tests. For example, it would be trivial to write a program that could pass an elementary student's math tests, but it would be rather ignorant to assume that a program passing that test was the same indication of that program's "understanding" of the underlying math that you would assume of a child who passed that same test.
throw_pm23 4y ago
> (1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) Why is it funny? We most certainly do exactly that. In particular in this forum where we interact via text input-output, but also in real life when we ask each other questions, etc. Sure, there are also gestures and other channels of communication but text input/output is a big part.
elixxir123 4y ago
> If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Let's see, ask gpt: Human: what is a probable meaning of the words "quick! fire!", and in which circumstances should it occur? ChatGPT: "Quick! Fire!" is an expression that may be used in a few different contexts, depending on the situation. Here are a few possible meanings: A command to shoot a gun or firearm: "Quick! Fire!" could be used as a command to instruct someone to shoot a gun, particularly in a military or law enforcement context. A call for help in the event of a fire: "Quick! Fire!" could be used to alert others to a fire that has broken out and to call for immediate assistance in putting it out or evacuating people from the area. A warning of danger or urgency: "Quick! Fire!" could be used to alert someone to a dangerous situation or to indicate that something urgent needs to be done right away. In general, the expression "Quick! Fire!" would be used in situations that require immediate action or attention to address an urgent or dangerous situation. So it seems that with a little more context chatGPT can understand what you mean with those words.
postultimate 4y ago
When GPT mentions ice-cream, it does so because it was in the corpus. When it occurred in the corpus, it was as a reference to actual ice-cream. So GPT has just as much intentionality as you do. You might claim that you've eaten ice-cream, and that that makes a difference. But if we assume that your senses aren't lying about what your senses do, then what they do is produce information - indications of difference without any indication of what it's a difference of. That puts you in the same epistemic position GPT is in. GPT knows just as much about ice-cream as you do.
- mjburgess 4y ago
  Saying "I like ice-cream" has obvious conditions under which the speaker means it. ChatGPT cannot meet those conditions. It lacks the capacity to like, indeed, to intend to say anything. ChatGPT cannot communicate. No act of text generation its engaged in counts as communication: it does not mean to say anything.
- imtringued 4y ago
  Let us construct IceCreamGPT. We take a corpus of text written by people who like ice cream and have provably demonstrated their joy while eating it. We then fine tune GPT 3.5 and the resulting model is called IceCreamGPT. Does IceCreamGPT like ice cream or is it only seemingly liking ice cream? It obviously likes ice cream, since it shares the same intentionality as humans responsible for the training data. Now do the same with people who don't like ice cream but lie and write that they like ice cream. The performance of the second model is identical to the first model. Does this mean IceCreamGPT2 likes ice cream? Of course not, IceCreamGPT2 doesn't like ice cream despite it saying it likes ice cream! We know it doesn't like ice cream because it has the same intentionality as the humans responsible for its training data. Now we have entered a magic world in which anything can mean anything.
  - postultimate 4y ago
    No, this is just question-begging by treating GPT's access to the world as being external to it, but your own as being part of you. If we fix this by treating your senses as external, then we can imagine a copy of you with its senses rewired so that artichokes* taste like icecream (and vice-versa). (plus we lie to you about which is which.) The resulting imtringued2 is identical to you, but doesn't like ice cream despite it saying it likes ice cream. Just like IceCreamGPT2. * Or some equally disgusting "food".

krona 4y ago

ChatGPT has state and representation, clearly. But that's besides the point. It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve. All of this is a bit like saying a car isn't a horse because it doesn't have hooves. Yes, but so what?

mjburgess 4y ago
The question of whether it has a representational model of the terms it's using is open at the moment. I'm strongly inclined to deny it, but the argument on this is at least quite complex and subject to premises which are less certain than the following: It does not form a representational state prior to speaking which is the reason it speaks. In other-words, it does not say, "I like ice-cream" because it likes ice-cream. That's sufficient to invalidate the idea that it has any relevant capacities here. We even know, for example, that it randomly chooses unlikely words to make its output seem more reasonable: this mechanism alone invalidates it having a representational model. --- And on the former point: if we look at pretty much all plausible theories of meaning, the meaning of terms cannot be anything like what ChatGPT has access to. I think what ChatGPT has is something like "a statistically approximate model of word connotation". Whether a statistical approximation of part of meaning counts as "being meaningful" is a matter of debate. (I think it clearly doesnt.)
- yunwal 4y ago
  > It does not form a representational state prior to speaking which is the reason it speaks. The whole thing is a representational state, and the fact that it's identical at the start of every conversation is a design choice, not inherent to the way the model learns. > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. You also don't say "I like ice-cream" because you like ice cream. You say it because through years of evolution you've been taught to crave fat and sugar and other things in ice cream, which fuels your body, and helps you live and reproduce. The fact that this AI is optimizing for something else doesn't really prove anything. I also don't think that we have anything more than a "statistically approximate model" of anything outside of our own thoughts. The theory of gravity exists because we've seen objects fall from heights a million times, we're told by others we speak to that they experience the same thing. From this we can say that with 99.9999...% (enough 9s to be indistinguishable from 100) probability, the theory of gravity is true.
  - simple-thoughts 4y ago
    I would amend that there is valid reasoning other than empirical analysis, but also that chatgpt is capable of generating text demonstrating those types of reasoning.
ahahahahah 4y ago
> It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve That's wrong. It's been demonstrated to produce responses to question such that if a 9 year old child would produce those same responses we would assume that that child had a typical 9 year-olds theory of mind. Not even for the child does "passing" such a test demonstrate having such theory of mind, though given our understanding of how humans develop it's a pretty good signal. I don't know how someone could understand that and still think that chatgpt has demonstrated theory of mind. It is, in fact, the same for a lot of tests. For example, it would be trivial to write a program that could pass an elementary student's math tests, but it would be rather ignorant to assume that a program passing that test was the same indication of that program's "understanding" of the underlying math that you would assume of a child who passed that same test.

throw_pm23 4y ago

> (1) You realise that we don't investigate human understanding by looking at input-ouptut text correspondances? (This is quite a funny claim!) Why is it funny? We most certainly do exactly that. In particular in this forum where we interact via text input-output, but also in real life when we ask each other questions, etc. Sure, there are also gestures and other channels of communication but text input/output is a big part.

elixxir123 4y ago

> If I say, "quick! a fire!" it will not do anything which accords with showing an understanding of that. Let's see, ask gpt: Human: what is a probable meaning of the words "quick! fire!", and in which circumstances should it occur? ChatGPT: "Quick! Fire!" is an expression that may be used in a few different contexts, depending on the situation. Here are a few possible meanings: A command to shoot a gun or firearm: "Quick! Fire!" could be used as a command to instruct someone to shoot a gun, particularly in a military or law enforcement context. A call for help in the event of a fire: "Quick! Fire!" could be used to alert others to a fire that has broken out and to call for immediate assistance in putting it out or evacuating people from the area. A warning of danger or urgency: "Quick! Fire!" could be used to alert someone to a dangerous situation or to indicate that something urgent needs to be done right away. In general, the expression "Quick! Fire!" would be used in situations that require immediate action or attention to address an urgent or dangerous situation. So it seems that with a little more context chatGPT can understand what you mean with those words.

postultimate 4y ago

When GPT mentions ice-cream, it does so because it was in the corpus. When it occurred in the corpus, it was as a reference to actual ice-cream. So GPT has just as much intentionality as you do. You might claim that you've eaten ice-cream, and that that makes a difference. But if we assume that your senses aren't lying about what your senses do, then what they do is produce information - indications of difference without any indication of what it's a difference of. That puts you in the same epistemic position GPT is in. GPT knows just as much about ice-cream as you do.

mjburgess 4y ago
Saying "I like ice-cream" has obvious conditions under which the speaker means it. ChatGPT cannot meet those conditions. It lacks the capacity to like, indeed, to intend to say anything. ChatGPT cannot communicate. No act of text generation its engaged in counts as communication: it does not mean to say anything.
imtringued 4y ago
Let us construct IceCreamGPT. We take a corpus of text written by people who like ice cream and have provably demonstrated their joy while eating it. We then fine tune GPT 3.5 and the resulting model is called IceCreamGPT. Does IceCreamGPT like ice cream or is it only seemingly liking ice cream? It obviously likes ice cream, since it shares the same intentionality as humans responsible for the training data. Now do the same with people who don't like ice cream but lie and write that they like ice cream. The performance of the second model is identical to the first model. Does this mean IceCreamGPT2 likes ice cream? Of course not, IceCreamGPT2 doesn't like ice cream despite it saying it likes ice cream! We know it doesn't like ice cream because it has the same intentionality as the humans responsible for its training data. Now we have entered a magic world in which anything can mean anything.
- postultimate 4y ago
  No, this is just question-begging by treating GPT's access to the world as being external to it, but your own as being part of you. If we fix this by treating your senses as external, then we can imagine a copy of you with its senses rewired so that artichokes* taste like icecream (and vice-versa). (plus we lie to you about which is which.) The resulting imtringued2 is identical to you, but doesn't like ice cream despite it saying it likes ice cream. Just like IceCreamGPT2. * Or some equally disgusting "food".