5 ms·

> Is chatGPT a mathematical model of consciousness? Unless we reduce models of consciousness to purely stochastic processes, this is trivially false. And no on

by dvt 4y ago

> Is chatGPT a mathematical model of consciousness? Unless we reduce models of consciousness to purely stochastic processes, this is trivially false. And no one seriously reduces consciousness to probabilistic pattern recognition. > We know it doesn't represent the whole experience... It doesn't represent any experience. ChatGPT doesn't "experience" things as conscious things do. It's not even comparable to the experience of a simple invertebrate, for example.

Lyapunov_Lover 4y ago
That's the (hard) problem of qualia. You can't say for sure that ChatGPT doesn't experience anything. And I can't say for sure that you do, either. It's trivial to dismiss it on the basis of "well obviously it's like that"—far more difficult is to say something about why it's not like that. I think consciousness has to be the product of an algorithmic process. I think this process can be summed up as "prediction". I don't think there's some secret quantum/soul/magic aspect to it. Which is why I think it's plausible that transformer models may capture some aspect of what we refer to as consciousness, though I don't quite buy it at a gut level. You say that "it's not even comparable to the experience of a simple invertebrate," but that's just you writing down what your gut is telling you. What is the experience of a simple invertebrate like, exactly?
- vba616 4y ago
  >You can't say for sure that ChatGPT doesn't experience anything. And I can't say for sure that you do, either. The question rattling around in my brain tonight is - if some, many, people don't develop a theory of mind as young children, how many of the people around me are like this? Is it just the philosophers? It seems of much more moment than the essence of ChatGPT.
  - messe 4y ago
    Given how many people in this thread don’t seem to grasp the idea of qualia, and think the hard problem of consciousness to be easy, I’m not entirely unconvinced I’m surrounded by a bunch of imperfect p-zombies.
    - vba616 4y ago
      People without a theory of mind are not p-zombies. They are people who see others as p-zombies. Given that, I'm not sure how to interpret your comment as it seems to describe an invalid inference.
kilgnad 4y ago
>Unless we reduce models of consciousness to purely stochastic processes, this is trivially false. And no one seriously reduces consciousness to probabilistic pattern recognition. Most things in this universe have mathematical models that can describe it. We have models for virtually everything. Thus it is reasonable to say such a model also exists for consciousness. To say what you said definitively without even knowing the meaning of the word consciousness is premature. You don't have enough knowledge to say anything about this. >It doesn't represent any experience. ChatGPT doesn't "experience" things as conscious things do. It's not even comparable to the experience of a simple invertebrate, for example. Again you don't know what you're talking about. Nobody can define what consciousness is and nobody fully understands the emergent effects of chatGPT or the human brain. To say what you say with conviction is therefore empty. You don't actually know because nobody knows.
- dvt 4y ago
  > Again you don't know what you're talking about. I think that's a bit uncharitable. You're acting like people haven't been grappling with these problems for thousands of years. Searle, for example, makes a pretty good argument why symbol-processing systems will never produce consciousness (you'd have to prove that the Chinese room is conscious, etc.). It's clear, for example, that a toaster doesn't have any conscious experiences, but a dog does. To say that "nobody fully understands" the human brain is throwing out the baby with the bathwater. ChatGPT is not conscious any more than my (very complex) Swiss wristwatch is.
  - kilgnad 4y ago
    Philosophers grapple with the problem the same way Greeks used mythology to grapple with the problem of reality. Thunder is not thrown by some god named Zeus. and the impossibility of a mathematical model describing consciousness cannot be explained away by a Chinese room argument that has tons and tons of qualitative holes in it. If an atom can be described by a mathematical equation that means so can groups of atoms. If groups of atoms can be described math and if a neuron is simply a complex group of atoms then a neuron can be described by an equation. If neurons can be described by an equation then so can groups of neurons. If the brain is a group of neurons then the brain can be described by a mathematical equation. If consciousness is a physical phenomena produced by a mathematical brain then consciousness is a mathematical model. The only way you can believe that consciousness cannot ever by described by math is if you see some break in the previous chain of logic. Most likely it's in the linking of consciousness entirely with the physical brain. Likely you believe in a soul or something if you don't agree with the logic. Which just means you're using mythology and concepts outside of science to derive your conclusions. Again, lighting is not thrown down from the sky by a god named Zeus. It's clear a toaster is not conscious. But what chatGPT is, is not clear. Placing it in a category where we aren't sure is reasonable and also it's a category agreed upon by many people. It would be a bit uncharitable to demand that your categorization of chatGPT is more correct then other peoples. Even you must be be aware of the ambiguity associated with chatGPT despite your stubborn stance to the contrary.
sdenton4 4y ago
Consciousness apparently arose from evolution, which is a funny process of optimization under stochastic conditions. (There's also a good amount of pattern recognition involved.) Why is it that optimizing for survival can lead to consciousness, but not predictive coding? Nothing here is trivial.
plutonorm 4y ago
While some people believe that consciousness cannot be reduced to probabilistic pattern recognition, I strongly disagree. In my view, the AIXI model is an excellent model of intelligence, as it can mathematically define a notion of Pareto optimal behavior across all possible environments. This model relies on compression at its core, specifically the minimum description length principle. Moreover, large language models (LLMs), like GPT-3, are designed to learn how to predict, and prediction is the dual of compression. As explained in this StackExchange post (https://cs.stackexchange.com/questions/130655/is-prediction-the-same-as-compression#:~:text=Prediction%20can%20be%20used%20to%20compress%2C%20and%20compression,can%20be%20used%20to%20compress%20and%20vice%20versa https://cs.stackexchange.com/questions/130655/is-prediction-...), prediction can be used to compress data, and compression can be used to make predictions. Given that LLMs are trained on vast amounts of data and can predict patterns within that data, I believe that they contain all the necessary components to be considered an intelligent system. Therefore, I firmly believe that consciousness can indeed be reduced to probabilistic pattern recognition, and that LLMs provide a compelling example of this phenomenon in action.
- prmph 4y ago
  Your viewpoint is one I consistently have to push against. Computation is not necessarily experience. I can't emphasize enough how fundamental understanding this is. It seems some just can't grasp this concept. Would you say a program that echoes to the screen the responses of a hidden human is intelligent? Viewed as a black box, it would certainly seem so. But does it have any internal experience and understanding of anything? Of course this is a trivial experiment, but the point of it is to show, in the most absurdly clear terms, that producing plausible outputs !== being intelligent, much less having a conscious experience. This example also shows that the perceived sophistication of the input-output transformation is immaterial, unless you want to say consciousness requires a certain internal architecture for computation, in which case I would ask: what architecture is that? No matter the sophistication of an algorithm, it's computation lies on a different axis to that of consciousness. You might say human compute because they are conscious, not conscious because they compute. Another way to understand this: Let's say you touch a very hot object. You skin, in conjunction with your brain, senses and computes the temperature of the object. But, there is no essential reason why that should lead to an experience of pain, rather than a sensation similar to color, which generally does not produce pain.
naasking 4y ago
> And no one seriously reduces consciousness to probabilistic pattern recognition. All eliminative materialists disagree. To them, consciousness is mechanistic in some way, so why not stochastic pattern matching which can be made Turing complete quite easily. For instance: https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119 > It doesn't represent any experience. ChatGPT doesn't "experience" things as conscious things do. You don't know that because we don't have a mechanistic understanding of "experience".

by dvt 4y ago

Lyapunov_Lover 4y ago

That's the (hard) problem of qualia. You can't say for sure that ChatGPT doesn't experience anything. And I can't say for sure that you do, either. It's trivial to dismiss it on the basis of "well obviously it's like that"—far more difficult is to say something about why it's not like that. I think consciousness has to be the product of an algorithmic process. I think this process can be summed up as "prediction". I don't think there's some secret quantum/soul/magic aspect to it. Which is why I think it's plausible that transformer models may capture some aspect of what we refer to as consciousness, though I don't quite buy it at a gut level. You say that "it's not even comparable to the experience of a simple invertebrate," but that's just you writing down what your gut is telling you. What is the experience of a simple invertebrate like, exactly?

vba616 4y ago
>You can't say for sure that ChatGPT doesn't experience anything. And I can't say for sure that you do, either. The question rattling around in my brain tonight is - if some, many, people don't develop a theory of mind as young children, how many of the people around me are like this? Is it just the philosophers? It seems of much more moment than the essence of ChatGPT.
- messe 4y ago
  Given how many people in this thread don’t seem to grasp the idea of qualia, and think the hard problem of consciousness to be easy, I’m not entirely unconvinced I’m surrounded by a bunch of imperfect p-zombies.
  - vba616 4y ago
    People without a theory of mind are not p-zombies. They are people who see others as p-zombies. Given that, I'm not sure how to interpret your comment as it seems to describe an invalid inference.

kilgnad 4y ago

>Unless we reduce models of consciousness to purely stochastic processes, this is trivially false. And no one seriously reduces consciousness to probabilistic pattern recognition. Most things in this universe have mathematical models that can describe it. We have models for virtually everything. Thus it is reasonable to say such a model also exists for consciousness. To say what you said definitively without even knowing the meaning of the word consciousness is premature. You don't have enough knowledge to say anything about this. >It doesn't represent any experience. ChatGPT doesn't "experience" things as conscious things do. It's not even comparable to the experience of a simple invertebrate, for example. Again you don't know what you're talking about. Nobody can define what consciousness is and nobody fully understands the emergent effects of chatGPT or the human brain. To say what you say with conviction is therefore empty. You don't actually know because nobody knows.

dvt 4y ago
> Again you don't know what you're talking about. I think that's a bit uncharitable. You're acting like people haven't been grappling with these problems for thousands of years. Searle, for example, makes a pretty good argument why symbol-processing systems will never produce consciousness (you'd have to prove that the Chinese room is conscious, etc.). It's clear, for example, that a toaster doesn't have any conscious experiences, but a dog does. To say that "nobody fully understands" the human brain is throwing out the baby with the bathwater. ChatGPT is not conscious any more than my (very complex) Swiss wristwatch is.
- kilgnad 4y ago
  Philosophers grapple with the problem the same way Greeks used mythology to grapple with the problem of reality. Thunder is not thrown by some god named Zeus. and the impossibility of a mathematical model describing consciousness cannot be explained away by a Chinese room argument that has tons and tons of qualitative holes in it. If an atom can be described by a mathematical equation that means so can groups of atoms. If groups of atoms can be described math and if a neuron is simply a complex group of atoms then a neuron can be described by an equation. If neurons can be described by an equation then so can groups of neurons. If the brain is a group of neurons then the brain can be described by a mathematical equation. If consciousness is a physical phenomena produced by a mathematical brain then consciousness is a mathematical model. The only way you can believe that consciousness cannot ever by described by math is if you see some break in the previous chain of logic. Most likely it's in the linking of consciousness entirely with the physical brain. Likely you believe in a soul or something if you don't agree with the logic. Which just means you're using mythology and concepts outside of science to derive your conclusions. Again, lighting is not thrown down from the sky by a god named Zeus. It's clear a toaster is not conscious. But what chatGPT is, is not clear. Placing it in a category where we aren't sure is reasonable and also it's a category agreed upon by many people. It would be a bit uncharitable to demand that your categorization of chatGPT is more correct then other peoples. Even you must be be aware of the ambiguity associated with chatGPT despite your stubborn stance to the contrary.

sdenton4 4y ago

Consciousness apparently arose from evolution, which is a funny process of optimization under stochastic conditions. (There's also a good amount of pattern recognition involved.) Why is it that optimizing for survival can lead to consciousness, but not predictive coding? Nothing here is trivial.

plutonorm 4y ago

While some people believe that consciousness cannot be reduced to probabilistic pattern recognition, I strongly disagree. In my view, the AIXI model is an excellent model of intelligence, as it can mathematically define a notion of Pareto optimal behavior across all possible environments. This model relies on compression at its core, specifically the minimum description length principle. Moreover, large language models (LLMs), like GPT-3, are designed to learn how to predict, and prediction is the dual of compression. As explained in this StackExchange post (https://cs.stackexchange.com/questions/130655/is-prediction-the-same-as-compression#:~:text=Prediction%20can%20be%20used%20to%20compress%2C%20and%20compression,can%20be%20used%20to%20compress%20and%20vice%20versa https://cs.stackexchange.com/questions/130655/is-prediction-...), prediction can be used to compress data, and compression can be used to make predictions. Given that LLMs are trained on vast amounts of data and can predict patterns within that data, I believe that they contain all the necessary components to be considered an intelligent system. Therefore, I firmly believe that consciousness can indeed be reduced to probabilistic pattern recognition, and that LLMs provide a compelling example of this phenomenon in action.

prmph 4y ago
Your viewpoint is one I consistently have to push against. Computation is not necessarily experience. I can't emphasize enough how fundamental understanding this is. It seems some just can't grasp this concept. Would you say a program that echoes to the screen the responses of a hidden human is intelligent? Viewed as a black box, it would certainly seem so. But does it have any internal experience and understanding of anything? Of course this is a trivial experiment, but the point of it is to show, in the most absurdly clear terms, that producing plausible outputs !== being intelligent, much less having a conscious experience. This example also shows that the perceived sophistication of the input-output transformation is immaterial, unless you want to say consciousness requires a certain internal architecture for computation, in which case I would ask: what architecture is that? No matter the sophistication of an algorithm, it's computation lies on a different axis to that of consciousness. You might say human compute because they are conscious, not conscious because they compute. Another way to understand this: Let's say you touch a very hot object. You skin, in conjunction with your brain, senses and computes the temperature of the object. But, there is no essential reason why that should lead to an experience of pain, rather than a sensation similar to color, which generally does not produce pain.

naasking 4y ago

> And no one seriously reduces consciousness to probabilistic pattern recognition. All eliminative materialists disagree. To them, consciousness is mechanistic in some way, so why not stochastic pattern matching which can be made Turing complete quite easily. For instance: https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119 > It doesn't represent any experience. ChatGPT doesn't "experience" things as conscious things do. You don't know that because we don't have a mechanistic understanding of "experience".