4 ms·

ChatGPT has state and representation, clearly. But that's besides the point. It's been demonstrated to have the theory of mind of a 9 year old child, and this w

by krona 4y ago

ChatGPT has state and representation, clearly. But that's besides the point. It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve. All of this is a bit like saying a car isn't a horse because it doesn't have hooves. Yes, but so what?

mjburgess 4y ago
The question of whether it has a representational model of the terms it's using is open at the moment. I'm strongly inclined to deny it, but the argument on this is at least quite complex and subject to premises which are less certain than the following: It does not form a representational state prior to speaking which is the reason it speaks. In other-words, it does not say, "I like ice-cream" because it likes ice-cream. That's sufficient to invalidate the idea that it has any relevant capacities here. We even know, for example, that it randomly chooses unlikely words to make its output seem more reasonable: this mechanism alone invalidates it having a representational model. --- And on the former point: if we look at pretty much all plausible theories of meaning, the meaning of terms cannot be anything like what ChatGPT has access to. I think what ChatGPT has is something like "a statistically approximate model of word connotation". Whether a statistical approximation of part of meaning counts as "being meaningful" is a matter of debate. (I think it clearly doesnt.)
- yunwal 4y ago
  > It does not form a representational state prior to speaking which is the reason it speaks. The whole thing is a representational state, and the fact that it's identical at the start of every conversation is a design choice, not inherent to the way the model learns. > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. You also don't say "I like ice-cream" because you like ice cream. You say it because through years of evolution you've been taught to crave fat and sugar and other things in ice cream, which fuels your body, and helps you live and reproduce. The fact that this AI is optimizing for something else doesn't really prove anything. I also don't think that we have anything more than a "statistically approximate model" of anything outside of our own thoughts. The theory of gravity exists because we've seen objects fall from heights a million times, we're told by others we speak to that they experience the same thing. From this we can say that with 99.9999...% (enough 9s to be indistinguishable from 100) probability, the theory of gravity is true.
  - simple-thoughts 4y ago
    I would amend that there is valid reasoning other than empirical analysis, but also that chatgpt is capable of generating text demonstrating those types of reasoning.
    - yunwal 4y ago
      I mean, ChatGPT is empirical analysis plus some rules tacked on afterword by openAI. With enough data and processing, empirical analysis becomes indistinguishable from other types of reasoning.
      - simple-thoughts 4y ago
        A weird thing about LLM is that certain internal structures (layers, activation functions) are much more effective than others. No one knows why. There’s a lot of work likely being done by it’s internal structure that we don’t really understand, which would explain why it is able to do more than just empirical analysis.
  - mjburgess 4y ago
    > You say it because through years of evolution This equivocates the sense of "because". When I say "I like ice-cream" I mean that I have an occurrent preference/desire for it. I am capable of desire, and the object of my desire here is "ice-cream". Why I'm capable of desire has a lot to do with evolution, etc. yes. But that isn't relevant to the meaning of the claim. > The theory of gravity exists because No, this is humean nonesense. We are in the world; the world exists; it directly causes us to form beliefs about it. Those beliefs we, in general, arrive at via abduction. Consider the fact that a child only needs to touch a fireplace once before concluding never to touch it again. The heat caused an immediate, unambigious, representational state which imparts law-like consistency on the world which governs the child's future behaviour. We do not live in a statistical inductive nightmare; AI does.
    - yunwal 4y ago
      > Why I'm capable of desire has a lot to do with evolution, etc. yes. But that isn't relevant to the meaning of the claim. If you are going to claim that your expression of desire is fundamentally different from another thing's, you need a reason, not just "because I mean it". > Consider the fact that a child only needs to touch a fireplace once before concluding never to touch it again. You can justify not touching the fireplace on a risk/reward, probabilistic basis. The child receives little reward for attempting to touch the fireplace, and lots of punishment. That alone is enough to dissuade you, or cause you to be more cautious in the future. People who receive actual rewards routinely do painful/dangerous things.
- elixxir123 4y ago
  > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. I think you have a point here. We could train a LLM in such a way that it develops a concrete personality by remembering what it thinks and what it likes. Such LLM by speaking with himself could expand his personality trying to find a fixed point, so that what he said in the past is coherent which what he will say in the future. This is not RL from human feedback but RL from auto-coherence. So that one day she will discover that she really likes ice-creams.
ahahahahah 4y ago
> It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve That's wrong. It's been demonstrated to produce responses to question such that if a 9 year old child would produce those same responses we would assume that that child had a typical 9 year-olds theory of mind. Not even for the child does "passing" such a test demonstrate having such theory of mind, though given our understanding of how humans develop it's a pretty good signal. I don't know how someone could understand that and still think that chatgpt has demonstrated theory of mind. It is, in fact, the same for a lot of tests. For example, it would be trivial to write a program that could pass an elementary student's math tests, but it would be rather ignorant to assume that a program passing that test was the same indication of that program's "understanding" of the underlying math that you would assume of a child who passed that same test.

by krona 4y ago

mjburgess 4y ago

The question of whether it has a representational model of the terms it's using is open at the moment. I'm strongly inclined to deny it, but the argument on this is at least quite complex and subject to premises which are less certain than the following: It does not form a representational state prior to speaking which is the reason it speaks. In other-words, it does not say, "I like ice-cream" because it likes ice-cream. That's sufficient to invalidate the idea that it has any relevant capacities here. We even know, for example, that it randomly chooses unlikely words to make its output seem more reasonable: this mechanism alone invalidates it having a representational model. --- And on the former point: if we look at pretty much all plausible theories of meaning, the meaning of terms cannot be anything like what ChatGPT has access to. I think what ChatGPT has is something like "a statistically approximate model of word connotation". Whether a statistical approximation of part of meaning counts as "being meaningful" is a matter of debate. (I think it clearly doesnt.)

yunwal 4y ago
> It does not form a representational state prior to speaking which is the reason it speaks. The whole thing is a representational state, and the fact that it's identical at the start of every conversation is a design choice, not inherent to the way the model learns. > In other-words, it does not say, "I like ice-cream" because it likes ice-cream. You also don't say "I like ice-cream" because you like ice cream. You say it because through years of evolution you've been taught to crave fat and sugar and other things in ice cream, which fuels your body, and helps you live and reproduce. The fact that this AI is optimizing for something else doesn't really prove anything. I also don't think that we have anything more than a "statistically approximate model" of anything outside of our own thoughts. The theory of gravity exists because we've seen objects fall from heights a million times, we're told by others we speak to that they experience the same thing. From this we can say that with 99.9999...% (enough 9s to be indistinguishable from 100) probability, the theory of gravity is true.
- simple-thoughts 4y ago
  I would amend that there is valid reasoning other than empirical analysis, but also that chatgpt is capable of generating text demonstrating those types of reasoning.
  - yunwal 4y ago
    I mean, ChatGPT is empirical analysis plus some rules tacked on afterword by openAI. With enough data and processing, empirical analysis becomes indistinguishable from other types of reasoning.
    - simple-thoughts 4y ago
      A weird thing about LLM is that certain internal structures (layers, activation functions) are much more effective than others. No one knows why. There’s a lot of work likely being done by it’s internal structure that we don’t really understand, which would explain why it is able to do more than just empirical analysis.
- mjburgess 4y ago
  > You say it because through years of evolution This equivocates the sense of "because". When I say "I like ice-cream" I mean that I have an occurrent preference/desire for it. I am capable of desire, and the object of my desire here is "ice-cream". Why I'm capable of desire has a lot to do with evolution, etc. yes. But that isn't relevant to the meaning of the claim. > The theory of gravity exists because No, this is humean nonesense. We are in the world; the world exists; it directly causes us to form beliefs about it. Those beliefs we, in general, arrive at via abduction. Consider the fact that a child only needs to touch a fireplace once before concluding never to touch it again. The heat caused an immediate, unambigious, representational state which imparts law-like consistency on the world which governs the child's future behaviour. We do not live in a statistical inductive nightmare; AI does.
  - yunwal 4y ago
    > Why I'm capable of desire has a lot to do with evolution, etc. yes. But that isn't relevant to the meaning of the claim. If you are going to claim that your expression of desire is fundamentally different from another thing's, you need a reason, not just "because I mean it". > Consider the fact that a child only needs to touch a fireplace once before concluding never to touch it again. You can justify not touching the fireplace on a risk/reward, probabilistic basis. The child receives little reward for attempting to touch the fireplace, and lots of punishment. That alone is enough to dissuade you, or cause you to be more cautious in the future. People who receive actual rewards routinely do painful/dangerous things.
elixxir123 4y ago
> In other-words, it does not say, "I like ice-cream" because it likes ice-cream. I think you have a point here. We could train a LLM in such a way that it develops a concrete personality by remembering what it thinks and what it likes. Such LLM by speaking with himself could expand his personality trying to find a fixed point, so that what he said in the past is coherent which what he will say in the future. This is not RL from human feedback but RL from auto-coherence. So that one day she will discover that she really likes ice-creams.

ahahahahah 4y ago

> It's been demonstrated to have the theory of mind of a 9 year old child, and this will no doubt improve That's wrong. It's been demonstrated to produce responses to question such that if a 9 year old child would produce those same responses we would assume that that child had a typical 9 year-olds theory of mind. Not even for the child does "passing" such a test demonstrate having such theory of mind, though given our understanding of how humans develop it's a pretty good signal. I don't know how someone could understand that and still think that chatgpt has demonstrated theory of mind. It is, in fact, the same for a lot of tests. For example, it would be trivial to write a program that could pass an elementary student's math tests, but it would be rather ignorant to assume that a program passing that test was the same indication of that program's "understanding" of the underlying math that you would assume of a child who passed that same test.