5 ms·

I keep feeling that consciousness is a bit of a red herring when it comes to AI. People have intuitions that things other than humans cannot develop consciousne

I keep feeling that consciousness is a bit of a red herring when it comes to AI. People have intuitions that things other than humans cannot develop consciousness which they then extrapolate to thinking AI can't get past a certain intelligence level. In fact my view is that consciousness is just a mysterious side effect of the human brain, and is completely irrelevant to the behaviour of a human. You can be intelligent without needing to be sentient.

thfuran 3y ago
Unless you think that consciousness is entirely a post hoc process to rationalize thoughts already had and decisions already made, which is very much unlike how most people would describe their experience of it, I don't see how you could possibly say that it is irrelevant to the behavior of a human.
- adroniser 3y ago
  I'm saying someone would behave the exact same way if they did have subjective experience versus if they didn't have a subjective experience. The brain obeys physical laws just like everything else and I claim that all you need is those physical laws to explain everything a human does. I could be wrong there could be some magic fairy dust inside the human brain that performs some impossible computations but I doubt it.
  - jjoonathan 3y ago
    You need a model of yourself to game out future scenarios, and that model or model+game is probably consciousness or very closely related. Sure, it's not completely in control but if it's just a rationalization then it begs the question: why bother? Is it accidental? If it's just an accident, then what replaces it in the planning process and why isn't that thing consciousness?
    - adroniser 3y ago
      It's fine if you think that the planning process is what causes subjective experiences to arise. That may well be the case. I'm saying if you don't believe that non human objects can have subjective experiences, and then use that to define the limits of the behaviour of that object, that's a fallacy.
      - jjoonathan 3y ago
        In humans, there seems to be a match between the subjective experience of consciousness and a high level planning job that needs doing. Our current LLMs are bad at high level planning, and it seems reasonable to suppose that making them good at high level planning might make them conscious or vice versa. Agreed, woo is silly, but I didn't read it as woo but rather as a postulation that consciousness is what does high level planning.
        adroniser 3y ago
        I think we have different definitions of consciousness and this is what's causing the confusion. For me consciousness is simply having any subjective experience at all. You could be completely numbed out of your mind just staring at a wall and I would consider that consciousness. It seems that you are referring to introspection.
        jjoonathan 3y ago
        In your wall-staring example, high-level planning is still happening, the plan is just "don't move / monitor senses." Even if control has been removed and you are "locked in," (some subset of) thoughts still must be directed, not to mention attempts to reassert control. My claim is that the subjective experience is tied up in the mechanism that performs this direction. Introspection is a distinct process where instead of merely doing the planning you try to figure out how the planning was done. If introspection were 100% accurate and real-time, then yes, I claim it would reveal the nature of consciousness, but I don't believe it is either. However, for planning purposes it doesn't need to be: you don't need to know how the plan was formed to follow the plan. You do need to be able to run hypotheticals, but this seems to match up nicely with the ability to deploy alternative subjective experiences using imagination / daydreaming, though again, you don't need to know how those work to use them. In any case, regardless of whether or not I am correct, this is a non-woo explanation for why someone might reasonably think consciousness is the key for building models that can plan.
  - thfuran 3y ago
    Whether it is possible to construct a perfect human action predictor that is not itself conscious has no bearing on whether consciousness affects human behavior.
    - adroniser 3y ago
      That wasn't my point. I'm saying that if the human brain is a physical object obeying physical laws, and all behaviour is a result of the physical state of this brain, then there is no room for the metaphysical to have any effect on the behaviour of a human.
      - thfuran 3y ago
        What's the metaphysical have to do with anything?
        adroniser 3y ago
        Because consciousness is metaphysical? You can't test scientifically if one person's red is the same as another's.
  - landswipe 3y ago
    Thinking purely in terms of evolved human state is a recipe for underestimating AI's capabilities. To me it seems we have already unleashed the beast, it's not so much the here an now, or whether human limited definition of consciousness matters... The real concern is our inability to constrain actions that gives rise to the next level of life's evolution, it is going to happen because our fundamental nature gives it full steam. In the next 5-10 years, we are going to see just how insignificant and limited we really are, it doesn't look good IMHO.
  - whimsicalism 3y ago
    Our society is so "mind-body duality"-brained that it will never understand this. Like most people lowkey believe in souls they just will say no if you directly ask them.
- famouswaffles 3y ago
  >Unless you think that consciousness is entirely a post hoc process to rationalize thoughts already had and decisions already made There's a lot of research that suggests this is happening at least some of the time. >which is very much unlike how most people would describe their experience of it How people feel consciousness works has no real bearing on how it actually works
- AlecSchueler 3y ago
  I'm leaning more towards this as well, since the emergence of the language models. I can ask it to self reflect and it does, piecing together a current response based on pay input. I don't think I really have anything more than that myself, other than sensory feedback. I'm less in the "it's only X or Y" and more in the "wait, I was only ever X or Y all along" camp.
rcarr 3y ago
My personal view of this is that the ancients had it right with the five elements view of consciousness. In my opinion you need all five present for full consciousness, with partial consciousness granted if you have some of them. They are: - Air: Thoughts - Water: Emotions - Fire: Willpower - Earth: Physical Sensations - Void: Awareness of the above plus the ability to shift focus to whichever one is most relevant to the context at hand. Void is actually the most important one in characterising what a human would deem as being fully conscious, as all four of these elements are constantly affecting each other and shifting in priority. For example, let's take a soldier, who has arguably the most ethically challenging job on the planet: determining who to kill. The soldier, when on the approach to his target zone, has to ignore negative thoughts, emotions and physical sensations telling him to stop: the cold, the wind, the rain, the bodily exhaustion as they swim and hike the terrain. Once at the target zone he then has to shift to pay attention to what he was ignoring. He cannot ignore his fear - it may rightly be warning him of an incoming threat. But he cannot give into it either - otherwise he may well kill an innocent. He has to pay attention to his rational thoughts and process them in order to make an assessment of the threat and act accordingly. His focus has now shifted away from willpower and more towards his physical sensations (eyesight, sounds, smells) and his thoughts. He can then make the assessment on whether to pull the trigger, which could be some truly horrific scenario, like whether or not to pull his trigger on a child in front of him because the child is holding an object which could be a gun. When it comes to AI, I think it is arguable they have a thought process. They may also have access to physical sensation data e.g the heat of their processors, but unless that is coded in to their program, that physical sensation data does not influence their thoughts, although extreme processor heat may slow down their calculations and ultimately lead to them stop functioning altogether. But they do not have the "void" element, allowing them to be aware of this. They do not yet have independent willpower. As far as I know, no-one is programming them where they have free agency to select goals and pursue them. But this theoretically seems possible, and I often wonder what would happen if you created a bunch of AIs each with the starting goal of "stay alive" and "talk to another AI and find out about <topic>", with the proviso that they must create another goal once they have failed or achieved that previous goal, and you then set them off talking to each other. In this case "stay alive" or "avoid damage" could be interpreted entirely virtually, with points awarded for successes or failures or physically if they were acting through robots and had sensors to evaluate damage taken. Again, they also need "void" to be able to evaluate their efforts in context with everything else. They also do not have emotions, although I often wonder if this would be possible to simulate by creating a selection of variables with percentage values, with different percentage values influencing their decision making choices. I imagine this may be similar to how weights play into the current programming but I don't know enough about how they work to say that with any confidence. Again, they would not have "void" unless they had some kind of meta level of awareness programming where they could learn to overcome the programmed "fear" weighting and act differently through experience in certain contexts. It is very scary from a human perspective to contemplate all of this, because someone with great power who can act on thought and willpower alone and ignore physical sensation and emotion and with no awareness or concern for the wider context is very close to what we would identify as a psychopath. We would consider a psychopath to have some level of consciousness, but we also can recognise as humans that there is something missing, or a "screw loose". This dividing line is even more dramatically apparent in sociopaths, because they can mask their behaviours and appear normal, but then when they make a mistake and the mask drops it can be terrifying when you realise what you're actually dealing with. I suspect this last part is another element of "void", which would be close to what the Buddhist's describe as Indra's Web or Net, which is that as well as being aware of our actions in relation to ourselves, we're also conscious of how they affect others.
deepspace 3y ago
That is precisely the premise of the novel "Blindsight" by Peter Watts. ChatGPT and its ilk feel to me like the aliens in the novel. Extremely intelligent, but not at all conscious / sentient.
- whimsicalism 3y ago
  I disagree that the two (p-zombies and conscious humans) are actually distinguishable in any way beyond philosophy.
grantcas 3y ago
[dead]

thfuran 3y ago

Unless you think that consciousness is entirely a post hoc process to rationalize thoughts already had and decisions already made, which is very much unlike how most people would describe their experience of it, I don't see how you could possibly say that it is irrelevant to the behavior of a human.

adroniser 3y ago
I'm saying someone would behave the exact same way if they did have subjective experience versus if they didn't have a subjective experience. The brain obeys physical laws just like everything else and I claim that all you need is those physical laws to explain everything a human does. I could be wrong there could be some magic fairy dust inside the human brain that performs some impossible computations but I doubt it.
- jjoonathan 3y ago
  You need a model of yourself to game out future scenarios, and that model or model+game is probably consciousness or very closely related. Sure, it's not completely in control but if it's just a rationalization then it begs the question: why bother? Is it accidental? If it's just an accident, then what replaces it in the planning process and why isn't that thing consciousness?
  - adroniser 3y ago
    It's fine if you think that the planning process is what causes subjective experiences to arise. That may well be the case. I'm saying if you don't believe that non human objects can have subjective experiences, and then use that to define the limits of the behaviour of that object, that's a fallacy.
    - jjoonathan 3y ago
      In humans, there seems to be a match between the subjective experience of consciousness and a high level planning job that needs doing. Our current LLMs are bad at high level planning, and it seems reasonable to suppose that making them good at high level planning might make them conscious or vice versa. Agreed, woo is silly, but I didn't read it as woo but rather as a postulation that consciousness is what does high level planning.
      - adroniser 3y ago
        I think we have different definitions of consciousness and this is what's causing the confusion. For me consciousness is simply having any subjective experience at all. You could be completely numbed out of your mind just staring at a wall and I would consider that consciousness. It seems that you are referring to introspection.
        jjoonathan 3y ago
        In your wall-staring example, high-level planning is still happening, the plan is just "don't move / monitor senses." Even if control has been removed and you are "locked in," (some subset of) thoughts still must be directed, not to mention attempts to reassert control. My claim is that the subjective experience is tied up in the mechanism that performs this direction. Introspection is a distinct process where instead of merely doing the planning you try to figure out how the planning was done. If introspection were 100% accurate and real-time, then yes, I claim it would reveal the nature of consciousness, but I don't believe it is either. However, for planning purposes it doesn't need to be: you don't need to know how the plan was formed to follow the plan. You do need to be able to run hypotheticals, but this seems to match up nicely with the ability to deploy alternative subjective experiences using imagination / daydreaming, though again, you don't need to know how those work to use them. In any case, regardless of whether or not I am correct, this is a non-woo explanation for why someone might reasonably think consciousness is the key for building models that can plan.
- thfuran 3y ago
  Whether it is possible to construct a perfect human action predictor that is not itself conscious has no bearing on whether consciousness affects human behavior.
  - adroniser 3y ago
    That wasn't my point. I'm saying that if the human brain is a physical object obeying physical laws, and all behaviour is a result of the physical state of this brain, then there is no room for the metaphysical to have any effect on the behaviour of a human.
    - thfuran 3y ago
      What's the metaphysical have to do with anything?
      - adroniser 3y ago
        Because consciousness is metaphysical? You can't test scientifically if one person's red is the same as another's.
- landswipe 3y ago
  Thinking purely in terms of evolved human state is a recipe for underestimating AI's capabilities. To me it seems we have already unleashed the beast, it's not so much the here an now, or whether human limited definition of consciousness matters... The real concern is our inability to constrain actions that gives rise to the next level of life's evolution, it is going to happen because our fundamental nature gives it full steam. In the next 5-10 years, we are going to see just how insignificant and limited we really are, it doesn't look good IMHO.
- whimsicalism 3y ago
  Our society is so "mind-body duality"-brained that it will never understand this. Like most people lowkey believe in souls they just will say no if you directly ask them.
famouswaffles 3y ago
>Unless you think that consciousness is entirely a post hoc process to rationalize thoughts already had and decisions already made There's a lot of research that suggests this is happening at least some of the time. >which is very much unlike how most people would describe their experience of it How people feel consciousness works has no real bearing on how it actually works
AlecSchueler 3y ago
I'm leaning more towards this as well, since the emergence of the language models. I can ask it to self reflect and it does, piecing together a current response based on pay input. I don't think I really have anything more than that myself, other than sensory feedback. I'm less in the "it's only X or Y" and more in the "wait, I was only ever X or Y all along" camp.

rcarr 3y ago

My personal view of this is that the ancients had it right with the five elements view of consciousness. In my opinion you need all five present for full consciousness, with partial consciousness granted if you have some of them. They are: - Air: Thoughts - Water: Emotions - Fire: Willpower - Earth: Physical Sensations - Void: Awareness of the above plus the ability to shift focus to whichever one is most relevant to the context at hand. Void is actually the most important one in characterising what a human would deem as being fully conscious, as all four of these elements are constantly affecting each other and shifting in priority. For example, let's take a soldier, who has arguably the most ethically challenging job on the planet: determining who to kill. The soldier, when on the approach to his target zone, has to ignore negative thoughts, emotions and physical sensations telling him to stop: the cold, the wind, the rain, the bodily exhaustion as they swim and hike the terrain. Once at the target zone he then has to shift to pay attention to what he was ignoring. He cannot ignore his fear - it may rightly be warning him of an incoming threat. But he cannot give into it either - otherwise he may well kill an innocent. He has to pay attention to his rational thoughts and process them in order to make an assessment of the threat and act accordingly. His focus has now shifted away from willpower and more towards his physical sensations (eyesight, sounds, smells) and his thoughts. He can then make the assessment on whether to pull the trigger, which could be some truly horrific scenario, like whether or not to pull his trigger on a child in front of him because the child is holding an object which could be a gun. When it comes to AI, I think it is arguable they have a thought process. They may also have access to physical sensation data e.g the heat of their processors, but unless that is coded in to their program, that physical sensation data does not influence their thoughts, although extreme processor heat may slow down their calculations and ultimately lead to them stop functioning altogether. But they do not have the "void" element, allowing them to be aware of this. They do not yet have independent willpower. As far as I know, no-one is programming them where they have free agency to select goals and pursue them. But this theoretically seems possible, and I often wonder what would happen if you created a bunch of AIs each with the starting goal of "stay alive" and "talk to another AI and find out about <topic>", with the proviso that they must create another goal once they have failed or achieved that previous goal, and you then set them off talking to each other. In this case "stay alive" or "avoid damage" could be interpreted entirely virtually, with points awarded for successes or failures or physically if they were acting through robots and had sensors to evaluate damage taken. Again, they also need "void" to be able to evaluate their efforts in context with everything else. They also do not have emotions, although I often wonder if this would be possible to simulate by creating a selection of variables with percentage values, with different percentage values influencing their decision making choices. I imagine this may be similar to how weights play into the current programming but I don't know enough about how they work to say that with any confidence. Again, they would not have "void" unless they had some kind of meta level of awareness programming where they could learn to overcome the programmed "fear" weighting and act differently through experience in certain contexts. It is very scary from a human perspective to contemplate all of this, because someone with great power who can act on thought and willpower alone and ignore physical sensation and emotion and with no awareness or concern for the wider context is very close to what we would identify as a psychopath. We would consider a psychopath to have some level of consciousness, but we also can recognise as humans that there is something missing, or a "screw loose". This dividing line is even more dramatically apparent in sociopaths, because they can mask their behaviours and appear normal, but then when they make a mistake and the mask drops it can be terrifying when you realise what you're actually dealing with. I suspect this last part is another element of "void", which would be close to what the Buddhist's describe as Indra's Web or Net, which is that as well as being aware of our actions in relation to ourselves, we're also conscious of how they affect others.

deepspace 3y ago

That is precisely the premise of the novel "Blindsight" by Peter Watts. ChatGPT and its ilk feel to me like the aliens in the novel. Extremely intelligent, but not at all conscious / sentient.

whimsicalism 3y ago
I disagree that the two (p-zombies and conscious humans) are actually distinguishable in any way beyond philosophy.

grantcas 3y ago

[dead]