4 ms·

> LLMs and AGI seem poised to give us a huge and expanding space in which to discover how consciousness works How? My own intuition is that LLMs will contribu

> LLMs and AGI seem poised to give us a huge and expanding space in which to discover how consciousness works How? My own intuition is that LLMs will contribute precisely nothing to the philosophy of consciousness. Philosophers have been considering the consequences of intelligent machines for some time.

randallsquared 3y ago
Philosophers have largely only been able to run thought experiments. While those can help clarify what we know, any apparently new knowledge generated thereby has to be confirmed by physical experiment. Physical experiments on human consciousness can only be done in very limited ways without ethical issues -- we don't have the technology to press the pause or undo buttons for human brains, and we don't have direct access to the full state of the brain. Assuming we develop (apparently) conscious minds on our current trajectory, there will be opportunities to run experiments in parallel, have real controls, and AGI researchers will, hypothetically, be able to self-introspect all the way down to the most basic level. You may object that there's no way to really know that another mind is conscious, and that no volume of physical evidence will prove it, and that will continue to be true even as we learn how it all works, in the same way that solipsism is both fully irrefutable and essentially ignored by anyone who wants to know more about how the world works.
- notamy 3y ago
  > Assuming we develop (apparently) conscious minds on our current trajectory, there will be opportunities to run experiments in parallel, have real controls, and AGI researchers will, hypothetically, be able to self-introspect all the way down to the most basic level. If a hypothetical AGI is truly believed to be conscious, how could it ever possibly be ethical to experiment on such a thing?
  - randallsquared 3y ago
    Consent plus the ability to reset make it at least possible to respect ethics in such experiments, unlike with biological organisms.
    - notamy 3y ago
      > Consent Which we could likely easily engineer into whatever system eventually exists, once that point is reached. I doubt anyone who makes such a thing would intentionally engineer it to NOT consent to their experimentation. > Consent plus the ability to reset For a hypothetical example: If I said I would perform <some psychological experiment> on you, but it's okay because I can force consent into you and then perfectly reset your brain and body back to how they were pre-experiment, would you still be okay with it? This may be a bit of an extreme way of looking at it, but the ethical concerns matter.
      - randallsquared 3y ago
        If you use force to obtain “consent”, then I would argue that it isn’t ethical and maybe that it’s a contradiction in terms. If you know how to create a person who would be enthusiastic about consenting, doing that doesn’t seem unethical. For example, after someone chooses to consent, copying them — I would argue that this is just creating a consenting participant, and not unethical (where that person is concerned).
- MaxBarraclough 3y ago
  What kind of experimental set-up do you have in mind? I'm skeptical it could be of deep interest to philosophers. It might tell you something about the current state of AI technology, but that would be it. Why would I care what answer the LLM gives to a question like, say, Are you conscious? A one-line Bash script can claim to be conscious, but offers no insight whatever into the nature of consciousness. We've known for some time that general anesthetics can suspend consciousness. 'Neural correlate' studies give a more detailed understanding of the relationship between brain activity and consciousness. These are of limited value to philosophy of consciousness, though. AI experimentation strikes me as having even less applicability.
crazygringo 3y ago
Exactly. Consciousness is about qualia -- the color of red, the texture of sand, and so forth. It's awareness. There's nothing we know now that suggests consciousness has anything whatsoever to do with intelligence. We are aware (or conscious) of our intelligent trains of thought (at least up to a certain point), in the same way we're aware of a pain in our knee. We are conscious of watching ourselves weigh the pros and cons of a decision we have to make. But there's nothing whatsoever to suggest that consciousness plays a role in intelligent thought itself, at least not yet. We know that we ourselves are both conscious and intelligent, but it's entirely possible that's a total coincidence.
- MaxBarraclough 3y ago
  > there's nothing whatsoever to suggest that consciousness plays a role in intelligent thought itself Well, there is in that both appear to arise from brain activity. The brain evolved in animals for just one reason: to drive or suppress muscular contraction activity in the individual. [0] Trees don't need this kind of information processing system (which is to say, intelligence), and so would have no use for a brain. It seems unlikely that consciousness is possible in the absence of any form of intelligence or information processing. The only kind of consciousness we know, appears to be 'along for the ride' as a byproduct of the brain activity. > there's nothing whatsoever to suggest that consciousness plays a role in intelligent thought itself I agree that it would be a reach to claim that all kinds of advanced information processing must necessarily give rise to consciousness. [0] Daniel Wolpert: The Real Reason for Brains https://www.youtube.com/watch?v=7s0CpRfyYp8&t=14 https://www.youtube.com/watch?v=7s0CpRfyYp8&t=14

randallsquared 3y ago

Philosophers have largely only been able to run thought experiments. While those can help clarify what we know, any apparently new knowledge generated thereby has to be confirmed by physical experiment. Physical experiments on human consciousness can only be done in very limited ways without ethical issues -- we don't have the technology to press the pause or undo buttons for human brains, and we don't have direct access to the full state of the brain. Assuming we develop (apparently) conscious minds on our current trajectory, there will be opportunities to run experiments in parallel, have real controls, and AGI researchers will, hypothetically, be able to self-introspect all the way down to the most basic level. You may object that there's no way to really know that another mind is conscious, and that no volume of physical evidence will prove it, and that will continue to be true even as we learn how it all works, in the same way that solipsism is both fully irrefutable and essentially ignored by anyone who wants to know more about how the world works.

notamy 3y ago
> Assuming we develop (apparently) conscious minds on our current trajectory, there will be opportunities to run experiments in parallel, have real controls, and AGI researchers will, hypothetically, be able to self-introspect all the way down to the most basic level. If a hypothetical AGI is truly believed to be conscious, how could it ever possibly be ethical to experiment on such a thing?
- randallsquared 3y ago
  Consent plus the ability to reset make it at least possible to respect ethics in such experiments, unlike with biological organisms.
  - notamy 3y ago
    > Consent Which we could likely easily engineer into whatever system eventually exists, once that point is reached. I doubt anyone who makes such a thing would intentionally engineer it to NOT consent to their experimentation. > Consent plus the ability to reset For a hypothetical example: If I said I would perform <some psychological experiment> on you, but it's okay because I can force consent into you and then perfectly reset your brain and body back to how they were pre-experiment, would you still be okay with it? This may be a bit of an extreme way of looking at it, but the ethical concerns matter.
    - randallsquared 3y ago
      If you use force to obtain “consent”, then I would argue that it isn’t ethical and maybe that it’s a contradiction in terms. If you know how to create a person who would be enthusiastic about consenting, doing that doesn’t seem unethical. For example, after someone chooses to consent, copying them — I would argue that this is just creating a consenting participant, and not unethical (where that person is concerned).
MaxBarraclough 3y ago
What kind of experimental set-up do you have in mind? I'm skeptical it could be of deep interest to philosophers. It might tell you something about the current state of AI technology, but that would be it. Why would I care what answer the LLM gives to a question like, say, Are you conscious? A one-line Bash script can claim to be conscious, but offers no insight whatever into the nature of consciousness. We've known for some time that general anesthetics can suspend consciousness. 'Neural correlate' studies give a more detailed understanding of the relationship between brain activity and consciousness. These are of limited value to philosophy of consciousness, though. AI experimentation strikes me as having even less applicability.

crazygringo 3y ago

Exactly. Consciousness is about qualia -- the color of red, the texture of sand, and so forth. It's awareness. There's nothing we know now that suggests consciousness has anything whatsoever to do with intelligence. We are aware (or conscious) of our intelligent trains of thought (at least up to a certain point), in the same way we're aware of a pain in our knee. We are conscious of watching ourselves weigh the pros and cons of a decision we have to make. But there's nothing whatsoever to suggest that consciousness plays a role in intelligent thought itself, at least not yet. We know that we ourselves are both conscious and intelligent, but it's entirely possible that's a total coincidence.

MaxBarraclough 3y ago
> there's nothing whatsoever to suggest that consciousness plays a role in intelligent thought itself Well, there is in that both appear to arise from brain activity. The brain evolved in animals for just one reason: to drive or suppress muscular contraction activity in the individual. [0] Trees don't need this kind of information processing system (which is to say, intelligence), and so would have no use for a brain. It seems unlikely that consciousness is possible in the absence of any form of intelligence or information processing. The only kind of consciousness we know, appears to be 'along for the ride' as a byproduct of the brain activity. > there's nothing whatsoever to suggest that consciousness plays a role in intelligent thought itself I agree that it would be a reach to claim that all kinds of advanced information processing must necessarily give rise to consciousness. [0] Daniel Wolpert: The Real Reason for Brains https://www.youtube.com/watch?v=7s0CpRfyYp8&t=14 https://www.youtube.com/watch?v=7s0CpRfyYp8&t=14