5 ms·

It's not clear what generates consciousness. Until we know for sure (e.g. A/B testing with humans who can report when they do and do not experience consciousne

It's not clear what generates consciousness. Until we know for sure (e.g. A/B testing with humans who can report when they do and do not experience consciousness in different neural configurations) then I think it's impossible to know what level of conscious experience large ML models have. Blindsight is an excellent book in its exploration of consciousness, but the speculative part is that a working sense of self isn't necessary for embodied intelligence (like the scramblers), which I tend to doubt. An agent without a model of itself will have difficulty planning actions; knowing how its outputs/manipulators are integrated into the rest of reality will be a minimum requirement to control them effectively. It is certainly possible that "self" or "I" will be absent; humans can already turn the ego off with drugs and still (mostly) function but they remain conscious.

mrcartmeneses 4y ago
People say this a lot online in this kind of debate but you can see which areas of the brain light up in people who are conscious and how that’s different from people who are asleep or in a persistent vegetative state. We have a very good idea of what consciousness is and it’s definitely not linear neural net with a discrete input and output. Having said that, the nerve that connects the eye to the brain could be considered similar to a modern neural net. It looks like we’re beginning to get the building blocks of consciousness together. But we don’t yet know how to combine the wave functions into a chorus necessary to achieve GI
- stormbrew 4y ago
  I think you're using a different meaning of the word conscious than the GP? You seem to be talking about awake, they're talking about a much more abstract idea. A person who is sleeping or in a vegetative state is not currently getting new inputs fed into some parts of their brain, so it's not surprising that their brain "lights up differently," nor does it imply anything about a piece of software that is getting new inputs that might be being integrated into its model (of course, a model that is trained and then repeatedly used without further integration is not in any way comparable to a brain). This more abstract idea of consciousness is definitely not a solved problem - people can't even manage to agree on whether non-human animals have it. And a lot of internet arguments for why this or that neural network can't be conscious probably also rule 5 year olds out of it too.
  - deleted 4y ago
    [deleted]
- benlivengood 4y ago
  We have fairly good ways of detecting human consciousness now, but not a lot of understanding of why physically or precisely which neural circuits produce conscious experience, which is the understanding we'd need to infer what sort of conscious experience is possible for ML models. We also can't reliably detect or predict whether a given animal is conscious or not (related to the problem of minimal viable neurons required for consciousness), and this is the biggest evidence I have that our study of consciousness is far from complete. The best we can do with humans (and maybe animals) is behaviorism and inference on our own personal consciousness at this point, with brain imaging to demonstrate at least gross prediction of consciousness in humans.
- visarga 4y ago
  > consciousness is definitely not linear neural net with a discrete input and output Just neural circuits are not going to be conscious by themselves, for one they need to learn concepts from the environment and those concepts shape the neural circuits. Thus the way they act shape how they develop. You can't separate consciousness from the environment where it develops. In other words it was not the neural net that was lacking, but the environment.
canjobear 4y ago
> humans who can report when they do and do not experience consciousness How can someone possibly report when they are not experiencing consciousness?
- benlivengood 4y ago
  > How can someone possibly report when they are not experiencing consciousness? By an absence of reporting it. If I sit at a desk getting my neurons moderated by testing equipment and say "I am conscious" every subjective second that I am experiencing consciousness then I could at least help narrow down when consciousness is lost. If I am simply unable to speak or respond at all, but still conscious, I would report that fact later. Only in the case of locked-in conscious awareness without later memory of the experience would this kind of experimental setup fail, and this is where brain imaging could probably help determine that everything except motor or memory neurons were active.
toomanydoubts 4y ago
>(e.g. A/B testing with humans who can report when they do and do not experience consciousness in different neural configurations) Doesn't that fall back to the old consciousnesses trap that nobody knows how to resolve? How do you know if the human reporting that he's conscious is actually conscious and not philosophical zombie? We don't know what generates consciousness because we don't know how to measure it, and if we can't measure it, we will always have to take the words of an seemingly conscious entity for it.
- benlivengood 4y ago
  Self-reporting is probably ideal, probably with a camera recording everything so I can judge my answers and behavior and compare it with my memory. But we don't really have a great handle on how much of our memory of conscious experience matches our real-time experience or even if the memory-formation process might "be" consciousness, so I admit there are blind spots in studying consciousness rigorously. I don't buy the philosophical zombie argument simply because consciousness does alter behavior. I wouldn't participate in this conversation the same way if I didn't experience consciousness. It would be more like vivid imagination (as apposed to moderate aphantasia) where I find it curious but don't have it. As in the novel, unconscious beings probably behave noticeably different. There are, apparently, some people who have a very reduced sense of consciousness. I know I have done and said things when I'm not (memorably) conscious, for example when half asleep or coming out of anesthesia, and my behavior has been altered according to witnesses. I wasn't quite "myself". I can also hyper-focus and reduce conscious awareness of my surroundings and of my own body and mind, but that still feels like I have an internal awareness and memory of the experience. I am fairly certain I would be able to tell if that is switched off for a time.
  - Scene_Cast2 4y ago
    There was this Google AI that made the news recently, it was trained to say that it's a conscious AI. However, most agree that it isn't conscious.
    - astrange 4y ago

mrcartmeneses 4y ago

People say this a lot online in this kind of debate but you can see which areas of the brain light up in people who are conscious and how that’s different from people who are asleep or in a persistent vegetative state. We have a very good idea of what consciousness is and it’s definitely not linear neural net with a discrete input and output. Having said that, the nerve that connects the eye to the brain could be considered similar to a modern neural net. It looks like we’re beginning to get the building blocks of consciousness together. But we don’t yet know how to combine the wave functions into a chorus necessary to achieve GI

stormbrew 4y ago
I think you're using a different meaning of the word conscious than the GP? You seem to be talking about awake, they're talking about a much more abstract idea. A person who is sleeping or in a vegetative state is not currently getting new inputs fed into some parts of their brain, so it's not surprising that their brain "lights up differently," nor does it imply anything about a piece of software that is getting new inputs that might be being integrated into its model (of course, a model that is trained and then repeatedly used without further integration is not in any way comparable to a brain). This more abstract idea of consciousness is definitely not a solved problem - people can't even manage to agree on whether non-human animals have it. And a lot of internet arguments for why this or that neural network can't be conscious probably also rule 5 year olds out of it too.
- deleted 4y ago
  [deleted]
benlivengood 4y ago
We have fairly good ways of detecting human consciousness now, but not a lot of understanding of why physically or precisely which neural circuits produce conscious experience, which is the understanding we'd need to infer what sort of conscious experience is possible for ML models. We also can't reliably detect or predict whether a given animal is conscious or not (related to the problem of minimal viable neurons required for consciousness), and this is the biggest evidence I have that our study of consciousness is far from complete. The best we can do with humans (and maybe animals) is behaviorism and inference on our own personal consciousness at this point, with brain imaging to demonstrate at least gross prediction of consciousness in humans.
visarga 4y ago
> consciousness is definitely not linear neural net with a discrete input and output Just neural circuits are not going to be conscious by themselves, for one they need to learn concepts from the environment and those concepts shape the neural circuits. Thus the way they act shape how they develop. You can't separate consciousness from the environment where it develops. In other words it was not the neural net that was lacking, but the environment.

canjobear 4y ago

> humans who can report when they do and do not experience consciousness How can someone possibly report when they are not experiencing consciousness?

benlivengood 4y ago
> How can someone possibly report when they are not experiencing consciousness? By an absence of reporting it. If I sit at a desk getting my neurons moderated by testing equipment and say "I am conscious" every subjective second that I am experiencing consciousness then I could at least help narrow down when consciousness is lost. If I am simply unable to speak or respond at all, but still conscious, I would report that fact later. Only in the case of locked-in conscious awareness without later memory of the experience would this kind of experimental setup fail, and this is where brain imaging could probably help determine that everything except motor or memory neurons were active.

toomanydoubts 4y ago

>(e.g. A/B testing with humans who can report when they do and do not experience consciousness in different neural configurations) Doesn't that fall back to the old consciousnesses trap that nobody knows how to resolve? How do you know if the human reporting that he's conscious is actually conscious and not philosophical zombie? We don't know what generates consciousness because we don't know how to measure it, and if we can't measure it, we will always have to take the words of an seemingly conscious entity for it.

benlivengood 4y ago
Self-reporting is probably ideal, probably with a camera recording everything so I can judge my answers and behavior and compare it with my memory. But we don't really have a great handle on how much of our memory of conscious experience matches our real-time experience or even if the memory-formation process might "be" consciousness, so I admit there are blind spots in studying consciousness rigorously. I don't buy the philosophical zombie argument simply because consciousness does alter behavior. I wouldn't participate in this conversation the same way if I didn't experience consciousness. It would be more like vivid imagination (as apposed to moderate aphantasia) where I find it curious but don't have it. As in the novel, unconscious beings probably behave noticeably different. There are, apparently, some people who have a very reduced sense of consciousness. I know I have done and said things when I'm not (memorably) conscious, for example when half asleep or coming out of anesthesia, and my behavior has been altered according to witnesses. I wasn't quite "myself". I can also hyper-focus and reduce conscious awareness of my surroundings and of my own body and mind, but that still feels like I have an internal awareness and memory of the experience. I am fairly certain I would be able to tell if that is switched off for a time.
- Scene_Cast2 4y ago
  There was this Google AI that made the news recently, it was trained to say that it's a conscious AI. However, most agree that it isn't conscious.
  - astrange 4y ago