4 ms·

> LLMs already meet the criteria for what most people technically define as consciousness. I strongly dispute this. Try "talking" to an LLM over time. It has n

> LLMs already meet the criteria for what most people technically define as consciousness. I strongly dispute this. Try "talking" to an LLM over time. It has no memory, no agency, no emotions, no desires, no pain, no sense of self. I think cats, dogs and certainly primates are far more conscious than these stochastic parrots. It do agree it's frightening how many people are fooled by a fancy autocomplete.

naasking 3y ago
> It has no memory, no agency, no emotions, no desires, no pain, no sense of self. It does have permanent memory, that's what training creates. It doesn't have agency. Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model of these phenomena such that we can see its presence in the human brain and see that LLMs do not have a comparable process. Of course you don't have such a model because it doesn't exist at this time. The problem is not that people have been fooled by stochastic parrots, it's that you've fooled yourself into thinking we understand more about the human mind than we actually do, and also possibly that humans are far more complex and special than might actually be the case, eg. that we are not basically stochastic parrots ourselves.
- ForHackernews 3y ago
  Speaking only for myself, I am not a stochastic parrot. I have a rich and complex inner life. But maybe you are a p-zombie and you are right that it's tricky for you to prove to me that you're not. ;) I don't necessarily think humans are so special (cf. my mention of other mammals) but biology is extraordinarily messy and complex. Computer people tend to handwave the details with crude, reductive analogies (DNA is like source code! Neurons are like circuits!) but the reality is so much more overwhelming. Every single thing affects every other thing. Everything is probabilistic. There are multiple interacting chemical and electrical communication pathways. Neuroanatomy varies from individual to individual and in the same individual over time. The brain can radically adapt in response to trauma in a way that no computer model does. Even at the level of individual neurons[0] (of which there are 2.25 billion in a dog's brain) there's an enormous amount of "processing" happening. This is not like a computer processing - but physical, biological processes occurring that integrate informational content. I get it, everyone is very impressed with LLMs and I'm open to the possibility that some future model might achieve consciousness (to say nothing of sentience) but computer people in particular are way, way too credulous of this stuff and ignorant of what the messy, malfunctioning wetware in their head is actually doing. [0] e.g. https://www.nature.com/articles/nrn2331 https://www.nature.com/articles/nrn2331
  - naasking 3y ago
    > I have a rich and complex inner life. You don't know that LLMs lack such a rich and complex inner life while they're running. > but computer people in particular are way, way too credulous of this stuff and ignorant of what the messy, malfunctioning wetware in their head is actually doing. Everything you mentioned is completely irrelevant to the argument, which is based on well known physical principles. Due to the Bekenstein Bound, I already know that you and your "rich and complex inner life" can be fully captured by a finite state automaton. Your protests that you are not a stochastic parrot based on hand-waving about your subjective experience, or vague assessments of how much processing is happening in a messy biological brain that has plenty of redundancies are simply not compelling. The fact is, we lack mechanistic models for any folk psychology concepts like "pain", "desires", "agency" and "sense of self", and so any claims that LLMs lack such things is conjecture at best. What I do know is that there's an impressively long history of wrong people who thought humans were special.
    - ForHackernews 3y ago
      > people who thought humans were special Literally the third time you've beaten this straw man. Keep going, though. All the other species of monkeys and I are enjoying it. > so any claims that LLMs lack such things is conjecture at best Holy shifting the burden of proof, Batman! It's incumbent on you to show that your fancy pattern-matching system is conscious. The null hypothesis is that your chatbot isn't conscious in any way that matters (I'll happily grant it a slightly-lower-than-thermostat[0] level of consciousness). I realize it's not easy for you to back up your bold claims, but all I see is a bunch of overconfident CS majors refusing to face up to the hard problem of consciousness[1]. Maybe you are an LLM, after all? Prove you're human: tell me a racist joke? (this question is a joke – something else LLMs are bad at) [0] http://romainbrette.fr/is-a-thermostat-conscious/ http://romainbrette.fr/is-a-thermostat-conscious/ [1] https://iep.utm.edu/hard-problem-of-conciousness/ https://iep.utm.edu/hard-problem-of-conciousness/
      - naasking 3y ago
        > Literally the third time you've beaten this straw man. Second actually. Are you an LLM since you're having trouble counting? > Holy shifting the burden of proof, Batman! It's incumbent on you to show that your fancy pattern-matching system is conscious. No, I never claimed they were conscious, I refuted the claim that we can say with confidence they are definitely not conscious, because such a claim requires knowledge humanity simply doesn't possess. Perhaps you should reread this thread if you're unclear about what I claimed. > I realize it's not easy for you to back up your bold claims, but all I see is a bunch of overconfident CS majors refusing to face up to the hard problem of consciousness[1] The hard problem is an alleged problem, there is no proof that it's real. All thought experiments that purport to show its existence (Mary's Room, p-zombies), have acceptable possible answers under physicalist interpretations, and some neuroscientists are now pursuing such theories [1]. [1] https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2015.00500/full https://www.frontiersin.org/journals/psychology/articles/10....
- staticman2 3y ago
  >>It does have permanent memory, that's what training creates. This is begging the question. By calling the output of the training "memory" you've proven it has "memory." I hereby rebut by calling the output of the training "parameters" thereby proving it has no "memory" only "parameters." >> Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model You probably don't go around saying it's unknown whether your desk chair has "desire, pain or sense of self". The fact tech enthusiasts go around rhetorically demanding "mechanistic models" to rebut their gushing claims about LLMs while not demanding "mechanistic models" while making gushing claims about desk chairs shows a lack of "mechanistic models" is not really the basis of their gushing claims.
  - naasking 3y ago
    > I hereby rebut by calling the output of the training "parameters" thereby proving it has no "memory" only "parameters." Since some types of parameters have the properties we ascribe to memory, then some types of parameters are memory. > You probably don't go around saying it's unknown whether your desk chair has "desire, pain or sense of self" I also don't go around having intelligent conversations with chairs. > The fact tech enthusiasts go around rhetorically demanding "mechanistic models" to rebut their gushing claims about LLMs while not demanding "mechanistic models" while making gushing claims about desk chairs shows a lack of "mechanistic models" is not really the basis of their gushing claims. I made no such claims, people like yourself who are so eager to dismiss the possibility that LLMs are intelligent or sentient are the ones making strong claims without justification. Edit: the fact that you are drawing an absurd equivalence between inert chairs and a system capable of intelligent conversation shows you haven't given this topic any serious thought. I suggest you do that and come back with a better analogy if you want to have a serious conversation.
    - ForHackernews 3y ago
      > I also don't go around having intelligent conversations with chairs. It seems like your functional theory of consciousness is "thing that produces plausible English output," which is, I might gently suggest, a bit flawed. Why is ChatGPT "conscious" for you but DALL-E is not? They're both big honking pattern-matching generators trained on lots of data. Why isn't ELIZA conscious, according to your theory? (or is it?)

naasking 3y ago

> It has no memory, no agency, no emotions, no desires, no pain, no sense of self. It does have permanent memory, that's what training creates. It doesn't have agency. Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model of these phenomena such that we can see its presence in the human brain and see that LLMs do not have a comparable process. Of course you don't have such a model because it doesn't exist at this time. The problem is not that people have been fooled by stochastic parrots, it's that you've fooled yourself into thinking we understand more about the human mind than we actually do, and also possibly that humans are far more complex and special than might actually be the case, eg. that we are not basically stochastic parrots ourselves.

ForHackernews 3y ago
Speaking only for myself, I am not a stochastic parrot. I have a rich and complex inner life. But maybe you are a p-zombie and you are right that it's tricky for you to prove to me that you're not. ;) I don't necessarily think humans are so special (cf. my mention of other mammals) but biology is extraordinarily messy and complex. Computer people tend to handwave the details with crude, reductive analogies (DNA is like source code! Neurons are like circuits!) but the reality is so much more overwhelming. Every single thing affects every other thing. Everything is probabilistic. There are multiple interacting chemical and electrical communication pathways. Neuroanatomy varies from individual to individual and in the same individual over time. The brain can radically adapt in response to trauma in a way that no computer model does. Even at the level of individual neurons[0] (of which there are 2.25 billion in a dog's brain) there's an enormous amount of "processing" happening. This is not like a computer processing - but physical, biological processes occurring that integrate informational content. I get it, everyone is very impressed with LLMs and I'm open to the possibility that some future model might achieve consciousness (to say nothing of sentience) but computer people in particular are way, way too credulous of this stuff and ignorant of what the messy, malfunctioning wetware in their head is actually doing. [0] e.g. https://www.nature.com/articles/nrn2331 https://www.nature.com/articles/nrn2331
- naasking 3y ago
  > I have a rich and complex inner life. You don't know that LLMs lack such a rich and complex inner life while they're running. > but computer people in particular are way, way too credulous of this stuff and ignorant of what the messy, malfunctioning wetware in their head is actually doing. Everything you mentioned is completely irrelevant to the argument, which is based on well known physical principles. Due to the Bekenstein Bound, I already know that you and your "rich and complex inner life" can be fully captured by a finite state automaton. Your protests that you are not a stochastic parrot based on hand-waving about your subjective experience, or vague assessments of how much processing is happening in a messy biological brain that has plenty of redundancies are simply not compelling. The fact is, we lack mechanistic models for any folk psychology concepts like "pain", "desires", "agency" and "sense of self", and so any claims that LLMs lack such things is conjecture at best. What I do know is that there's an impressively long history of wrong people who thought humans were special.
  - ForHackernews 3y ago
    > people who thought humans were special Literally the third time you've beaten this straw man. Keep going, though. All the other species of monkeys and I are enjoying it. > so any claims that LLMs lack such things is conjecture at best Holy shifting the burden of proof, Batman! It's incumbent on you to show that your fancy pattern-matching system is conscious. The null hypothesis is that your chatbot isn't conscious in any way that matters (I'll happily grant it a slightly-lower-than-thermostat[0] level of consciousness). I realize it's not easy for you to back up your bold claims, but all I see is a bunch of overconfident CS majors refusing to face up to the hard problem of consciousness[1]. Maybe you are an LLM, after all? Prove you're human: tell me a racist joke? (this question is a joke – something else LLMs are bad at) [0] http://romainbrette.fr/is-a-thermostat-conscious/ http://romainbrette.fr/is-a-thermostat-conscious/ [1] https://iep.utm.edu/hard-problem-of-conciousness/ https://iep.utm.edu/hard-problem-of-conciousness/
    - naasking 3y ago
      > Literally the third time you've beaten this straw man. Second actually. Are you an LLM since you're having trouble counting? > Holy shifting the burden of proof, Batman! It's incumbent on you to show that your fancy pattern-matching system is conscious. No, I never claimed they were conscious, I refuted the claim that we can say with confidence they are definitely not conscious, because such a claim requires knowledge humanity simply doesn't possess. Perhaps you should reread this thread if you're unclear about what I claimed. > I realize it's not easy for you to back up your bold claims, but all I see is a bunch of overconfident CS majors refusing to face up to the hard problem of consciousness[1] The hard problem is an alleged problem, there is no proof that it's real. All thought experiments that purport to show its existence (Mary's Room, p-zombies), have acceptable possible answers under physicalist interpretations, and some neuroscientists are now pursuing such theories [1]. [1] https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2015.00500/full https://www.frontiersin.org/journals/psychology/articles/10....
staticman2 3y ago
>>It does have permanent memory, that's what training creates. This is begging the question. By calling the output of the training "memory" you've proven it has "memory." I hereby rebut by calling the output of the training "parameters" thereby proving it has no "memory" only "parameters." >> Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model You probably don't go around saying it's unknown whether your desk chair has "desire, pain or sense of self". The fact tech enthusiasts go around rhetorically demanding "mechanistic models" to rebut their gushing claims about LLMs while not demanding "mechanistic models" while making gushing claims about desk chairs shows a lack of "mechanistic models" is not really the basis of their gushing claims.
- naasking 3y ago
  > I hereby rebut by calling the output of the training "parameters" thereby proving it has no "memory" only "parameters." Since some types of parameters have the properties we ascribe to memory, then some types of parameters are memory. > You probably don't go around saying it's unknown whether your desk chair has "desire, pain or sense of self" I also don't go around having intelligent conversations with chairs. > The fact tech enthusiasts go around rhetorically demanding "mechanistic models" to rebut their gushing claims about LLMs while not demanding "mechanistic models" while making gushing claims about desk chairs shows a lack of "mechanistic models" is not really the basis of their gushing claims. I made no such claims, people like yourself who are so eager to dismiss the possibility that LLMs are intelligent or sentient are the ones making strong claims without justification. Edit: the fact that you are drawing an absurd equivalence between inert chairs and a system capable of intelligent conversation shows you haven't given this topic any serious thought. I suggest you do that and come back with a better analogy if you want to have a serious conversation.
  - ForHackernews 3y ago
    > I also don't go around having intelligent conversations with chairs. It seems like your functional theory of consciousness is "thing that produces plausible English output," which is, I might gently suggest, a bit flawed. Why is ChatGPT "conscious" for you but DALL-E is not? They're both big honking pattern-matching generators trained on lots of data. Why isn't ELIZA conscious, according to your theory? (or is it?)