6 ms·

You sound like you don't actually understand anything about LLMs and are buying into the hype. They are not cognizant let alone conscious. They don't understand

You sound like you don't actually understand anything about LLMs and are buying into the hype. They are not cognizant let alone conscious. They don't understand anything. The tokens could be patterns of colored shapes with no actual meaning, only statistical distributions and nothing about how the LLMs work would change.

stevenhuang 2y ago
> The tokens could be patterns of colored shapes with no actual meaning, only statistical distributions and nothing about how the LLMs work would change. I can put your brain in a vat and stimulate your sensory neurons with a statistical distribution with no actual meaning, and nothing about how your brain works would change either. The LLM and your brain would attempt to interpret meaning with referent from training, and both would be confused at the information-free stimuli. Because during "training" in both cases, the stimuli received from the environment is structured and meaningful. So what's your point? By the way, pretty sure a neuroscientist with 20 years of ML experience has a deeper understanding of what "meaning" is than you do. Not to mention, your response reveals a significant ignorance of unresolved philosophical problems (hard problem of consciousness, what even is meaning) which you then use to incorrectly assume a foregone conclusion that whatever consciousness/meaning/reasoning is, LLMs must not have it. I'm partial to doubting LLMs as they are now have the magic sauce, but it's more that we don't actually know enough to say otherwise, so why state that we do know? We can't even say we know our own brains.
- greenthrow 2y ago
  Your response is nonsense. We don't know how consciousness arises from matter but we do have significant understandings about knowledge, reasoning, modeling, visualization, object permanence, etc. etc. that are significant parts of how the human mind works. And we know LLMs have none of these. The point of my colored shape example is that it is an illusion that there is anything resembling a mind inside an LLM. I thought that was obvious enough I didn't need to explain it further than I did. As far as the original commenter's credentials; there's lots of people who should know better but buy into hype and nonsense.
  - stevenhuang 2y ago
    > And we know LLMs have none of these. Go ahead and cite your sources. For every study claiming that LLMs lack these qualities, there are others that support and reinforce the connectionist model of how knowledge is encoded, and with other parallels to the human brain. So... it's inconclusive. It's bizarre why you so strongly insist otherwise when it's clear you are not informed. > The point of my colored shape example is that it is an illusion that there is anything resembling a mind inside an LLM And my example with subjecting a human brain through your procedure is to illustrate what a garbage experiment design it is. You wouldn't be able to tell there's a mind inside either. Both LLM and human brain mind would be confused. Both would "continue working" in the same way, trying to interpret meaning from meaningless stimulation. So you don't have a point to make, got it.
    - greenthrow 2y ago
      We know LLMs don't have those things prima facia because they fail at all of them constantly. We also know how they work, they are token predicters. That is all they are and all they can do. What can be accomplished with that is pretty cool, but humans love to imagine there is more going on when there isn't. Just like with Eliza. If you don't understand how your attempt to apply my colored shape analogy to a human brain is nonsensical I am not going to waste my time explaining it to you. I had a point, I made it, and apparently it escaped you.
      - stevenhuang 2y ago
        And if you don't see how the example with the human brain throws a wrench in your analogy, it would explain why you'd think it as nonsensical, as it's exactly of relevance. > We also know how they work, they are token predicters. That is all they are and all they can do Ah there it is, you've betrayed a deep lack of understanding in all relevant disciplines (neuroscience, cognition, information theory) required to even appreciate the many errors you've made here. You sure understand the subject matter and have nothing possibly to learn. Enjoy. https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
- southernplaces7 2y ago
  >I can put your brain in a vat and stimulate your sensory neurons with a statistical distribution with no actual meaning, and nothing about how your brain works would change either. The LLM and your brain would attempt to interpret meaning with referent from training, and both would be confused at the information-free stimuli. Because during "training" in both cases, the stimuli received from the environment is structured and meaningful. What an absurd response. Yes, you'd probably cause the human brain to start malfunctioning terribly at the form of consciousness it's well -accustomed to managing within the context of its normal physical substrate and environment. You'd be doing that (and thus degenerating it badly) because you removed it from that ancient context whose workings we still don't well understand. Your LLM on the other hand, has no context in which it shows such a level of cognitive capacity, higher-order reasoning, self direction and self awareness that we daily see humans to be capable of. >By the way, pretty sure a neuroscientist with 20 years of ML experience has a deeper understanding of what "meaning" is than you do. really? An appeal to authority? Many smart, educated people can still fall for utter nonsense and emotional attachment to bad ideas.
  - stevenhuang 2y ago
    > Your LLM on the other hand, has no context in which it shows such a level of cognitive capacity Yes, it will be confused as well, and for all outwards observable signs will fail to make sense of the stimuli, yet it will "aware" of its inability to understand, much like a human brain would. If you doubt that, open a new session and type some random tokens, you will get the answer that it's confused. Any other statement as to "consciousness" verges into the philosophical and unanswerable via empirical means. And ah, to frame it as an appeal to authority when the topic is precisely the subject of a neuroscientist's study. Sounds like you know a thing or two about nonsense and emotional attachment to bad ideas.
    - southernplaces7 2y ago
      You persist in talking nonsense. >Yes, it will be confused as well, and for all outwards observable signs will fail to make sense of the stimuli, yet it will "aware" of its inability to understand, much like a human brain would. >If you doubt that, open a new session and type some random tokens, you will get the answer that it's confused. There is no empirical evidence of any awareness whatsoever in any LLM, at all. Even their most immersed creators don't make such a claim. An LLM itself saying anything about awareness doesn't mean a thing. It's literally designed to mimic in such a way. And you speak of discussions of consciousness being about the philosophical and unanswerable? At least when talking about human awareness, one applies these ideas to minds that we personally as humans perceive to be aware and self-directed from our own experience (flawed as it is). You're applying the same notion to something that shows no evidence of awareness while then criticizing assumptions of consciousness in a human brain? Such a sloppy argument indeed does make appeals to authority necessary I suppose.

stevenhuang 2y ago

> The tokens could be patterns of colored shapes with no actual meaning, only statistical distributions and nothing about how the LLMs work would change. I can put your brain in a vat and stimulate your sensory neurons with a statistical distribution with no actual meaning, and nothing about how your brain works would change either. The LLM and your brain would attempt to interpret meaning with referent from training, and both would be confused at the information-free stimuli. Because during "training" in both cases, the stimuli received from the environment is structured and meaningful. So what's your point? By the way, pretty sure a neuroscientist with 20 years of ML experience has a deeper understanding of what "meaning" is than you do. Not to mention, your response reveals a significant ignorance of unresolved philosophical problems (hard problem of consciousness, what even is meaning) which you then use to incorrectly assume a foregone conclusion that whatever consciousness/meaning/reasoning is, LLMs must not have it. I'm partial to doubting LLMs as they are now have the magic sauce, but it's more that we don't actually know enough to say otherwise, so why state that we do know? We can't even say we know our own brains.

greenthrow 2y ago
Your response is nonsense. We don't know how consciousness arises from matter but we do have significant understandings about knowledge, reasoning, modeling, visualization, object permanence, etc. etc. that are significant parts of how the human mind works. And we know LLMs have none of these. The point of my colored shape example is that it is an illusion that there is anything resembling a mind inside an LLM. I thought that was obvious enough I didn't need to explain it further than I did. As far as the original commenter's credentials; there's lots of people who should know better but buy into hype and nonsense.
- stevenhuang 2y ago
  > And we know LLMs have none of these. Go ahead and cite your sources. For every study claiming that LLMs lack these qualities, there are others that support and reinforce the connectionist model of how knowledge is encoded, and with other parallels to the human brain. So... it's inconclusive. It's bizarre why you so strongly insist otherwise when it's clear you are not informed. > The point of my colored shape example is that it is an illusion that there is anything resembling a mind inside an LLM And my example with subjecting a human brain through your procedure is to illustrate what a garbage experiment design it is. You wouldn't be able to tell there's a mind inside either. Both LLM and human brain mind would be confused. Both would "continue working" in the same way, trying to interpret meaning from meaningless stimulation. So you don't have a point to make, got it.
  - greenthrow 2y ago
    We know LLMs don't have those things prima facia because they fail at all of them constantly. We also know how they work, they are token predicters. That is all they are and all they can do. What can be accomplished with that is pretty cool, but humans love to imagine there is more going on when there isn't. Just like with Eliza. If you don't understand how your attempt to apply my colored shape analogy to a human brain is nonsensical I am not going to waste my time explaining it to you. I had a point, I made it, and apparently it escaped you.
    - stevenhuang 2y ago
      And if you don't see how the example with the human brain throws a wrench in your analogy, it would explain why you'd think it as nonsensical, as it's exactly of relevance. > We also know how they work, they are token predicters. That is all they are and all they can do Ah there it is, you've betrayed a deep lack of understanding in all relevant disciplines (neuroscience, cognition, information theory) required to even appreciate the many errors you've made here. You sure understand the subject matter and have nothing possibly to learn. Enjoy. https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
southernplaces7 2y ago
>I can put your brain in a vat and stimulate your sensory neurons with a statistical distribution with no actual meaning, and nothing about how your brain works would change either. The LLM and your brain would attempt to interpret meaning with referent from training, and both would be confused at the information-free stimuli. Because during "training" in both cases, the stimuli received from the environment is structured and meaningful. What an absurd response. Yes, you'd probably cause the human brain to start malfunctioning terribly at the form of consciousness it's well -accustomed to managing within the context of its normal physical substrate and environment. You'd be doing that (and thus degenerating it badly) because you removed it from that ancient context whose workings we still don't well understand. Your LLM on the other hand, has no context in which it shows such a level of cognitive capacity, higher-order reasoning, self direction and self awareness that we daily see humans to be capable of. >By the way, pretty sure a neuroscientist with 20 years of ML experience has a deeper understanding of what "meaning" is than you do. really? An appeal to authority? Many smart, educated people can still fall for utter nonsense and emotional attachment to bad ideas.
- stevenhuang 2y ago
  > Your LLM on the other hand, has no context in which it shows such a level of cognitive capacity Yes, it will be confused as well, and for all outwards observable signs will fail to make sense of the stimuli, yet it will "aware" of its inability to understand, much like a human brain would. If you doubt that, open a new session and type some random tokens, you will get the answer that it's confused. Any other statement as to "consciousness" verges into the philosophical and unanswerable via empirical means. And ah, to frame it as an appeal to authority when the topic is precisely the subject of a neuroscientist's study. Sounds like you know a thing or two about nonsense and emotional attachment to bad ideas.
  - southernplaces7 2y ago
    You persist in talking nonsense. >Yes, it will be confused as well, and for all outwards observable signs will fail to make sense of the stimuli, yet it will "aware" of its inability to understand, much like a human brain would. >If you doubt that, open a new session and type some random tokens, you will get the answer that it's confused. There is no empirical evidence of any awareness whatsoever in any LLM, at all. Even their most immersed creators don't make such a claim. An LLM itself saying anything about awareness doesn't mean a thing. It's literally designed to mimic in such a way. And you speak of discussions of consciousness being about the philosophical and unanswerable? At least when talking about human awareness, one applies these ideas to minds that we personally as humans perceive to be aware and self-directed from our own experience (flawed as it is). You're applying the same notion to something that shows no evidence of awareness while then criticizing assumptions of consciousness in a human brain? Such a sloppy argument indeed does make appeals to authority necessary I suppose.