3 ms·

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from anyone else. It's always the software people! Consciousness is far from being fully understood but having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness. Margaret Wilsons paper from 2002 is a good read: https://link.springer.com/content/pdf/10.3758/BF03196322.pdf https://link.springer.com/content/pdf/10.3758/BF03196322.pdf peace

mirekrusin 3y ago
Are you saying that ie. paralyzed people don't have consciousness?
- bena 3y ago
  First of all, paralyzed people do have bodies. And sensorimotor functions. Second of all, it wouldn't matter if individually they did or did not. The species does and now our species has developed consciousness. It's part of the package. If you wanted a counterexample, you should look to plant life. There is some discussion on whether or not plant systems have a form on consciousness. But, then again, plants have bodies at the very least.
  - mirekrusin 3y ago
    What about people with amputations or missing body parts due to genetics - are they less conscious?
- zhynn 3y ago
  There is a spectrum between conscious and unconscious. You could say that under general anesthesia you are a 0/10 on the conscious scale, asleep is 1 or 2, just woken from sleep is maybe 3.... and up to a well rested, well fed, healthy sober adult human near the top of the scale. These are common sense notions of objective consciousness and they are testable and noncontroversial outside of a philosophy argument. Does this make sense as a rebuttal to your reductio argument?
  - mirekrusin 3y ago
    People dream under general anesthesia, which is a form of consciousness. I don't think ie. Stephen Hawking was on "lower spectrum of consciousness" either.
oh_sigh 3y ago
I suspect you don't know what OPs field of expertise is. I also doubt OP would disagree with that statement that the only conscious things we know of have a body and sensorimotor interactions with the environment.
Chabsff 3y ago
The befuddlement goes even farther for me. LLMs are, effectively, black-box systems that interface with the world via a stochastic parrot interface. You'd think that software engineers would be a group that easily understands how making radical assumptions about implementation details when looking at nothing but an interface is generally misguided. I'm not saying that there isn't a strong case to be made against LLMs being intelligent. It's pointing at the stochastic parrot as evidence enough in of itself that confuses me.
- throw0101a 3y ago
  > You'd think that […] As a stochastic parrot I'm unable to do that.
- lainga 3y ago
  Ah, but HN is a platform for not just any software engineer, but the entrepreneurial type.
  - NoGravitas 3y ago
    Which is to say, the type that will believe anything if it's currently easy to hype to VCs.
deleted 3y ago
[deleted]
fragmede 3y ago
To be fair, it's any of the exalted professions that the blessed extend their expertise to. Doctors, lawyers, software engineers. they (we) start with the notion that I'm a smart person, so from first principles, I can conquer the world. nevermind that that's an existing body of work, with their own practitioners to build off of.
danielmarkbruce 3y ago
Yeah, it's only software people. No one else has unfounded opinions. But... a parrot has a body. And sure, you'll say "they don't literally mean parrot".. but it's a vague term when you unpack it, and people saying "we are stochastic parrots" are also making a pretty vague comment (they clearly don't mean literally). Anyone who has a small child and understands LLMs is shocked by how much similar they seem to be when it comes to producing output.
- dekhn 3y ago
  The question is: does polly really want a cracker?
dekhn 3y ago
Some of us who believe that humans are at least partly statistical parrots have PhDs in relevant fields- for example, my PhD is in Biophysics, I've studied cognitive neuroscience and ML for decades, and while I think embodiment may very well be a necessary condition to reproduce the subjective experience of consciousness, I don't think "having a body and sensorimotor interactions are established as fundamental preconditions for cognition and in turn consciousness". Frankly I think that's an impractical question to answer. Instead, I work with the following idea: it seems not unlikely that we will, in the next decade or so, create non-embodied machine learning models which simply can't be told apart from a human (through a video chat-like interface). If you can do that, who really cares about whether it's conscious or not? I don't really think philosophy of the mind is that important here; instead, we should treat this as an engineering problem where we assume brains are subjectively conscious, but that's not a metric we are aiming for.
nix0n 3y ago
> software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise There's an XKCD about this behavior[0]. The title is actually "Physicists", but I also have seen it on HN (especially with psychology). [0]https://xkcd.com/793/ https://xkcd.com/793/
- User23 3y ago
  Well with psychology it’s more fair. Thanks to the replication crisis we can say with a straight face that psychologists aren’t even experts on psychology. As usual the demonstrable expertise in the field lies with the pragmatic types. For psychology that means salesmen, pickup artists, advertisers, conmen, propagandists, high school queen bees, and so on.
- dekhn 3y ago
  This is known as the "Why don't you just assume a spherical cow?"
hackinthebochs 3y ago
Embodiment is an intellectual dead end in explaining consciousness/sentience. Sure, its relevant to understanding human cognition as we are embodied entities, but it's not much relevant to consciousness as such. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The wide content of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our experiences. Our situatedness in the world is only relevant insofar as it entails the space of possible sensory experiences. Our sensory experience is the medium by which we learn about the external world. We learn of apples not because of the redness of the sensory experience, but because the pattern of red/not-red experience entails the shape of apples. Conscious experience provides the medium, modulations of which provide the information about features of the external world. It is analogous to how modulations of electromagnetic waves provides information about some distant information source. Understanding consciousness is an orthogonal matter to one's situatedness in the world, just like understanding electromagnetic waves is orthogonal to understanding the information source being modulated into them.
pzo 3y ago
> having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness. Does someone who is blind is not conscious then? How about when someone who is paralysed? or deaf? or someone with low IQ or mental illness? Stephen Hawking was paralysed most of his life but was a smart guy. If he was not only paralysed but also blind and deaf he would be smart guy still. I don't think AGI needs to have a body, sensorimotor interaction or even vision to be conscious. We need those for training - if you would be blind, paralysed, deaf from the beginning it would be hard for you learn anything and interact in any way. Machines have 6th sense that humans don't have - kind of a telepathy where they can exchange tokens/thoughts with different machines or humans much faster than we humans can type or speak.
- Teever 3y ago
  I've been thinking about this for a while now but I've been approaching it from the opposite direction. If we were attempting to put someone into some sort of Matrix like reality simulator but we lacked the technology to provide a perfect simulation what level of simulation would be 'good enough' that a human would consider it reality and be able to develop into something we could relate to? If you gave someone the Helen Keller level of experience, but with reduced tactile sensation, how much could you reduce that touch sensation before they wouldn't be like us?
  - GenericPoster 3y ago
    > If we were attempting to put someone into some sort of Matrix like reality simulator but we lacked the technology to provide a perfect simulation what level of simulation would be 'good enough' that a human would consider it reality and be able to develop into something we could relate to? Have you tried VR before? You really don't need perfect simulation to be fooled. Good enough is already here, albeit for a short amount of time.
PaulDavisThe1st 3y ago
From the first line of Wilson's paper: > There is a movement afoot in cognitive science to grant the body a central role in shaping the mind. It is far from true to say "having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness" It is a popular idea in some groups of people that study these questions. But there are many other similar groups of peopel studying these questions who do not agree with it, certainly not stated as strongly as you have put it here. Also, a reminder that HN readership and its commentariat, while dominated by SWE, is not limited to them.
findomgle 3y ago
[flagged]

mirekrusin 3y ago

Are you saying that ie. paralyzed people don't have consciousness?

bena 3y ago
First of all, paralyzed people do have bodies. And sensorimotor functions. Second of all, it wouldn't matter if individually they did or did not. The species does and now our species has developed consciousness. It's part of the package. If you wanted a counterexample, you should look to plant life. There is some discussion on whether or not plant systems have a form on consciousness. But, then again, plants have bodies at the very least.
- mirekrusin 3y ago
  What about people with amputations or missing body parts due to genetics - are they less conscious?
zhynn 3y ago
There is a spectrum between conscious and unconscious. You could say that under general anesthesia you are a 0/10 on the conscious scale, asleep is 1 or 2, just woken from sleep is maybe 3.... and up to a well rested, well fed, healthy sober adult human near the top of the scale. These are common sense notions of objective consciousness and they are testable and noncontroversial outside of a philosophy argument. Does this make sense as a rebuttal to your reductio argument?
- mirekrusin 3y ago
  People dream under general anesthesia, which is a form of consciousness. I don't think ie. Stephen Hawking was on "lower spectrum of consciousness" either.

oh_sigh 3y ago

I suspect you don't know what OPs field of expertise is. I also doubt OP would disagree with that statement that the only conscious things we know of have a body and sensorimotor interactions with the environment.

Chabsff 3y ago

The befuddlement goes even farther for me. LLMs are, effectively, black-box systems that interface with the world via a stochastic parrot interface. You'd think that software engineers would be a group that easily understands how making radical assumptions about implementation details when looking at nothing but an interface is generally misguided. I'm not saying that there isn't a strong case to be made against LLMs being intelligent. It's pointing at the stochastic parrot as evidence enough in of itself that confuses me.

throw0101a 3y ago
> You'd think that […] As a stochastic parrot I'm unable to do that.
lainga 3y ago
Ah, but HN is a platform for not just any software engineer, but the entrepreneurial type.
- NoGravitas 3y ago
  Which is to say, the type that will believe anything if it's currently easy to hype to VCs.

deleted 3y ago

[deleted]

fragmede 3y ago

To be fair, it's any of the exalted professions that the blessed extend their expertise to. Doctors, lawyers, software engineers. they (we) start with the notion that I'm a smart person, so from first principles, I can conquer the world. nevermind that that's an existing body of work, with their own practitioners to build off of.

danielmarkbruce 3y ago

Yeah, it's only software people. No one else has unfounded opinions. But... a parrot has a body. And sure, you'll say "they don't literally mean parrot".. but it's a vague term when you unpack it, and people saying "we are stochastic parrots" are also making a pretty vague comment (they clearly don't mean literally). Anyone who has a small child and understands LLMs is shocked by how much similar they seem to be when it comes to producing output.

dekhn 3y ago
The question is: does polly really want a cracker?

dekhn 3y ago

Some of us who believe that humans are at least partly statistical parrots have PhDs in relevant fields- for example, my PhD is in Biophysics, I've studied cognitive neuroscience and ML for decades, and while I think embodiment may very well be a necessary condition to reproduce the subjective experience of consciousness, I don't think "having a body and sensorimotor interactions are established as fundamental preconditions for cognition and in turn consciousness". Frankly I think that's an impractical question to answer. Instead, I work with the following idea: it seems not unlikely that we will, in the next decade or so, create non-embodied machine learning models which simply can't be told apart from a human (through a video chat-like interface). If you can do that, who really cares about whether it's conscious or not? I don't really think philosophy of the mind is that important here; instead, we should treat this as an engineering problem where we assume brains are subjectively conscious, but that's not a metric we are aiming for.

nix0n 3y ago

> software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise There's an XKCD about this behavior[0]. The title is actually "Physicists", but I also have seen it on HN (especially with psychology). [0]https://xkcd.com/793/ https://xkcd.com/793/

User23 3y ago
Well with psychology it’s more fair. Thanks to the replication crisis we can say with a straight face that psychologists aren’t even experts on psychology. As usual the demonstrable expertise in the field lies with the pragmatic types. For psychology that means salesmen, pickup artists, advertisers, conmen, propagandists, high school queen bees, and so on.
dekhn 3y ago
This is known as the "Why don't you just assume a spherical cow?"

hackinthebochs 3y ago

Embodiment is an intellectual dead end in explaining consciousness/sentience. Sure, its relevant to understanding human cognition as we are embodied entities, but it's not much relevant to consciousness as such. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The wide content of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our experiences. Our situatedness in the world is only relevant insofar as it entails the space of possible sensory experiences. Our sensory experience is the medium by which we learn about the external world. We learn of apples not because of the redness of the sensory experience, but because the pattern of red/not-red experience entails the shape of apples. Conscious experience provides the medium, modulations of which provide the information about features of the external world. It is analogous to how modulations of electromagnetic waves provides information about some distant information source. Understanding consciousness is an orthogonal matter to one's situatedness in the world, just like understanding electromagnetic waves is orthogonal to understanding the information source being modulated into them.

pzo 3y ago

> having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness. Does someone who is blind is not conscious then? How about when someone who is paralysed? or deaf? or someone with low IQ or mental illness? Stephen Hawking was paralysed most of his life but was a smart guy. If he was not only paralysed but also blind and deaf he would be smart guy still. I don't think AGI needs to have a body, sensorimotor interaction or even vision to be conscious. We need those for training - if you would be blind, paralysed, deaf from the beginning it would be hard for you learn anything and interact in any way. Machines have 6th sense that humans don't have - kind of a telepathy where they can exchange tokens/thoughts with different machines or humans much faster than we humans can type or speak.

Teever 3y ago
I've been thinking about this for a while now but I've been approaching it from the opposite direction. If we were attempting to put someone into some sort of Matrix like reality simulator but we lacked the technology to provide a perfect simulation what level of simulation would be 'good enough' that a human would consider it reality and be able to develop into something we could relate to? If you gave someone the Helen Keller level of experience, but with reduced tactile sensation, how much could you reduce that touch sensation before they wouldn't be like us?
- GenericPoster 3y ago
  > If we were attempting to put someone into some sort of Matrix like reality simulator but we lacked the technology to provide a perfect simulation what level of simulation would be 'good enough' that a human would consider it reality and be able to develop into something we could relate to? Have you tried VR before? You really don't need perfect simulation to be fooled. Good enough is already here, albeit for a short amount of time.

PaulDavisThe1st 3y ago

From the first line of Wilson's paper: > There is a movement afoot in cognitive science to grant the body a central role in shaping the mind. It is far from true to say "having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness" It is a popular idea in some groups of people that study these questions. But there are many other similar groups of peopel studying these questions who do not agree with it, certainly not stated as strongly as you have put it here. Also, a reminder that HN readership and its commentariat, while dominated by SWE, is not limited to them.

findomgle 3y ago

[flagged]