6 ms·

> Currently, the company is working to get its mini-brains—which so far are approaching the processing power of a dragonfly brain—to play the old Atari arcade g

> Currently, the company is working to get its mini-brains—which so far are approaching the processing power of a dragonfly brain—to play the old Atari arcade game Pong I'm surprised that no-one has discussed whether these systems could develop emergent qualia, and experience pain. No joke. Are there any ethical frameworks around this kind of research?

Sosh101 7y ago
I think that is a very valid concern, but how could this ever be assessed
- anonytrary 7y ago
  If something cannot be assessed, then are we still accountable for the result? Or, do we simply discard the assessment?
  - ganzuul 7y ago
    Human neural networks definitely are capable of suffering. These researchers might be re-inventing slavery.
    - collyw 7y ago
      They could give the artificial brains a bank account and pay them a salary.
    - battery_cowboy 7y ago
      Read "The Quantum Thief" by Hannu Rajaniemi
    - meowface 7y ago
      So could any future AI researcher, regardless of the substrate. I suspect this idea is so widespread at this point that these researchers and others are aware of this possibility and see the potential moral issues. I just think they believe such a thing is still decades (or longer) away, and I think they're probably right. Right now, just about everyone wildly sensationalizes things that appear to suggest conscious man-made entities, like the AIs that supposedly "invented their own language". I think we're so eager to see it even when there's nothing close to it that once it actually does happen, we'll realize it quickly and understand the implications. There is the possibility that it could somehow develop despite no external signs of it, but all we can do is try to be as cognizant of the possibility as we can.
  - austinjp 7y ago
    Or discard the project, since the key variable will remain forever unknowable.
colordrops 7y ago
Are only biological systems capable of experiencing qualia? What is to say that a silicon computer can't also have subjective experiences?
- Rexxar 7y ago
  Arguably, computers seems to suffer when there are be computations : they becomes hot and start to breathe a lot. That's a little irrational be I used to feel a little bad when I saw my old computer "suffering" when I was younger.
  - MaxBarraclough 7y ago
    Seems unlikely that's suffering, any more than petrol suffers. Animals (like us) suffer as part of our evolved aversion response. We have an in-built capacity for preference. Current-day computers lack this.
  - spyder 7y ago
    Just because you are hot and breath a lot it doesn't mean you are suffering, you are just working more. You start to suffer when you work over your limits, like when you overclock a computer too much and starts to glitch because some parts of it cannot handle it.
- collyw 7y ago
  That's the big question that science so far has no clue about.
- h0l0cube 7y ago
  I do sometimes entertain the idea of panpsychism (like perhaps a rock is conscious), and consciousness may not be substrate dependent, but I can be certain, by at least my own anecdata, that neurons give rise to qualia. The jury is still out on this, but a big-world network with some quantum computation could be enough. The 'hard problem of consciousness' is that we can't really know by observing a system whether it is conscious or not, or rather a sort of 'zombie' that has all the features resembling a conscious being without qualia.
taneq 7y ago
I don't think qualia are real in the general objective case, only in the subjective case. So nobody but 'you', whoever you are, can actually have them.
- s_gourichon 7y ago
  Ok let's rephrase then: can those mini-brains be a "you"?
  - dr1337 7y ago
    I guess if the stem-cells are taken from you as a donor then they would be genetically identical to you as much as twins are genetically identical individuals?
  - MaxBarraclough 7y ago
    Mandatory link to Wikipedia's article on Nagel's What Is It Like to Be a Bat? [0] > Nagel famously asserts that “an organism has conscious mental states if and only if there is something that it is like to be that organism—something it is like for the organism." [0] https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F
  - taneq 7y ago
    Of course... but so can any appropriately constructed control system, and so can any living organism which senses its surroundings to some degree.
- MaxBarraclough 7y ago
  Meaning what? That doesn't bring any light to the question of whether these things are capable of suffering. As someone else said, this is a hard problem to reason about.
  - CaptArmchair 7y ago
    This is the point where you end up in the realm of philosophy and phenomenology.
nnq 7y ago
> could develop emergent qualia If we were to admit there really is such thing as "qualia", there's not reason you wouldn't ask the same question about non-biological software or hardware systems... electronic hardware or biological hardware can have same computational qualities. But since OBJECTIVELY there's no such things as qualia (the concept only exist SUBJECTIVELY), it could only exist by definition for "a person/subject" eg., in this context, a neural network large and complex enough to get close to a human-like intelligence level! A dragonfly nervous system is probably simpler than some of the the largest artificial neural networks models...
- MaxBarraclough 7y ago
  I'm inclined to agree. To put it another way, then: what matters isn't the substrate (neurons vs transistors), but what you do with them. Going this route, we'd have to say that if you wrote a program that perfectly simulated the human brain, you've built a conscious system running on transistors. It also means that if you built a RISC-V processor based on a large number of neurons (say, as many as are in a human brain), you haven't built a conscious system, despite that it's neuron-based. I'm ignoring the obvious problems with building a RISC-V using neurons; they strike me as incidental detail, for our purposes here.
- h0l0cube 7y ago
  I've answered to these points in another comment. But I'll emphasize, that I know (and presume you know) within some minimum epistemic bound^, that we can suffer pain and that arises from our neurological systems. It can't be known what other systems/substrates could experience qualia, and even if they have the outward markers of consciousness, they could be 'zombies'. https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie ^ i.e., not in some adversarial simulation etc.
- eyeundersand 7y ago
  > A dragonfly nervous system is probably simpler than some of the largest artificial neural networks models. I would be extremely surprised if that were the case. Behavior of a single cell alone is unimaginably complex. Let alone an organ system, let alone the nervous system. I will agree with your Qualia point.
tartoran 7y ago
I am not an expert but from my understanding pain are signals perceived as what we subjectively call painful when something in our body/nervous system is damaged/inflamed or a system is out of whack. It is a very useful thing, it is trying to tell us something useful so we can act on it or just be aware of it. When something is out of whack permanently something in this system is malfunctioning. Without a body it is hard to describe the concept of pain and what it may feel to an emergent system with qualia properties. The closest I could think of is some kind of distress signal or base frequency change when the said system is under stress or wrong state and is perhaps used to self correct the system to a balanced state. If something is permanently painful then it ceases to be a reliable signal. Psychological pain or emotional pain or distress may come closer to what such a system could experience and but it could also be an emergent property of such a system but we don't fully understand how this works. If such a system has emotional capability then it should be able to experience it because pain is such a state of the emotional spectrum. No pain no joy - no oscillation/flat
totalZero 7y ago
We are all machines. The idea that one type of machine or another cannot feel "pain" is simply a rationalization for human behavior that causes harm to other organisms, like hunting and fishing and killing wasps. Pain is a sensation that drives an animal to quickly react to injurious situations. Consider a robot made with Brooks's subsumption architecture [1]. The basal level of behavior drive of the robot is its self-preservation. That level of behavior drive can halt all the higher-order behaviors to monopolize processing and locomotion. Imagine that this behavior drive is triggered when the ambient temperature approaches a level that can damage the battery, and attempts to move the robot to the lowest-temperature zone available. That is functionally no different than what happens to a person in a house fire. All we have done with the word "pain" is to set up an arbitrary delineation based on the hormonal/neuronal response that links our sensory apparatus to our behavioral drive. [1] https://people.csail.mit.edu/brooks/papers/representation.pdf https://people.csail.mit.edu/brooks/papers/representation.pd...
- tartoran 7y ago
  I agree with you, pain is a useful signal or it could be a systemic damage that wrongly sends these signals. However, when we, humans, refer to pain we clump all the unpleasant things we experience together. One such pain is the emotional /psychological pain and this type of pain is more likely to occur in such a system without a body. This is also a signal/state and is also useful to delineate it from another state. If such system is to experience joy it better know what pain is or otherwise wouldn't be able to delineate.
- ben_w 7y ago
  Careful, or you might use a definition of pain too broad to be useful. Plants, single-celled organisms, thermostats, and anti-lock braking systems meet your use of the word. Almost all of us know what it means to be in pain. It’s the hard to put into words subjective experience of suffering that is worth making ethical decisions about, not the autonomous reflex that doesn’t come with conciousness-is-a-poor-word-but-I-lack-a-better-alternative.
  - totalZero 7y ago
russdill 7y ago
We're already really good at building mini-brains that can play pong..and much more advanced games than pong. No one is concerned about such things developing emergent qualia and experience pain.

Sosh101 7y ago

I think that is a very valid concern, but how could this ever be assessed

anonytrary 7y ago
If something cannot be assessed, then are we still accountable for the result? Or, do we simply discard the assessment?
- ganzuul 7y ago
  Human neural networks definitely are capable of suffering. These researchers might be re-inventing slavery.
  - collyw 7y ago
    They could give the artificial brains a bank account and pay them a salary.
  - battery_cowboy 7y ago
    Read "The Quantum Thief" by Hannu Rajaniemi
  - meowface 7y ago
    So could any future AI researcher, regardless of the substrate. I suspect this idea is so widespread at this point that these researchers and others are aware of this possibility and see the potential moral issues. I just think they believe such a thing is still decades (or longer) away, and I think they're probably right. Right now, just about everyone wildly sensationalizes things that appear to suggest conscious man-made entities, like the AIs that supposedly "invented their own language". I think we're so eager to see it even when there's nothing close to it that once it actually does happen, we'll realize it quickly and understand the implications. There is the possibility that it could somehow develop despite no external signs of it, but all we can do is try to be as cognizant of the possibility as we can.
- austinjp 7y ago
  Or discard the project, since the key variable will remain forever unknowable.

colordrops 7y ago

Are only biological systems capable of experiencing qualia? What is to say that a silicon computer can't also have subjective experiences?

Rexxar 7y ago
Arguably, computers seems to suffer when there are be computations : they becomes hot and start to breathe a lot. That's a little irrational be I used to feel a little bad when I saw my old computer "suffering" when I was younger.
- MaxBarraclough 7y ago
  Seems unlikely that's suffering, any more than petrol suffers. Animals (like us) suffer as part of our evolved aversion response. We have an in-built capacity for preference. Current-day computers lack this.
- spyder 7y ago
  Just because you are hot and breath a lot it doesn't mean you are suffering, you are just working more. You start to suffer when you work over your limits, like when you overclock a computer too much and starts to glitch because some parts of it cannot handle it.
collyw 7y ago
That's the big question that science so far has no clue about.
h0l0cube 7y ago
I do sometimes entertain the idea of panpsychism (like perhaps a rock is conscious), and consciousness may not be substrate dependent, but I can be certain, by at least my own anecdata, that neurons give rise to qualia. The jury is still out on this, but a big-world network with some quantum computation could be enough. The 'hard problem of consciousness' is that we can't really know by observing a system whether it is conscious or not, or rather a sort of 'zombie' that has all the features resembling a conscious being without qualia.

taneq 7y ago

I don't think qualia are real in the general objective case, only in the subjective case. So nobody but 'you', whoever you are, can actually have them.

s_gourichon 7y ago
Ok let's rephrase then: can those mini-brains be a "you"?
- dr1337 7y ago
  I guess if the stem-cells are taken from you as a donor then they would be genetically identical to you as much as twins are genetically identical individuals?
- MaxBarraclough 7y ago
  Mandatory link to Wikipedia's article on Nagel's What Is It Like to Be a Bat? [0] > Nagel famously asserts that “an organism has conscious mental states if and only if there is something that it is like to be that organism—something it is like for the organism." [0] https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F
- taneq 7y ago
  Of course... but so can any appropriately constructed control system, and so can any living organism which senses its surroundings to some degree.
MaxBarraclough 7y ago
Meaning what? That doesn't bring any light to the question of whether these things are capable of suffering. As someone else said, this is a hard problem to reason about.
- CaptArmchair 7y ago
  This is the point where you end up in the realm of philosophy and phenomenology.

nnq 7y ago

> could develop emergent qualia If we were to admit there really is such thing as "qualia", there's not reason you wouldn't ask the same question about non-biological software or hardware systems... electronic hardware or biological hardware can have same computational qualities. But since OBJECTIVELY there's no such things as qualia (the concept only exist SUBJECTIVELY), it could only exist by definition for "a person/subject" eg., in this context, a neural network large and complex enough to get close to a human-like intelligence level! A dragonfly nervous system is probably simpler than some of the the largest artificial neural networks models...

MaxBarraclough 7y ago
I'm inclined to agree. To put it another way, then: what matters isn't the substrate (neurons vs transistors), but what you do with them. Going this route, we'd have to say that if you wrote a program that perfectly simulated the human brain, you've built a conscious system running on transistors. It also means that if you built a RISC-V processor based on a large number of neurons (say, as many as are in a human brain), you haven't built a conscious system, despite that it's neuron-based. I'm ignoring the obvious problems with building a RISC-V using neurons; they strike me as incidental detail, for our purposes here.
h0l0cube 7y ago
I've answered to these points in another comment. But I'll emphasize, that I know (and presume you know) within some minimum epistemic bound^, that we can suffer pain and that arises from our neurological systems. It can't be known what other systems/substrates could experience qualia, and even if they have the outward markers of consciousness, they could be 'zombies'. https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie ^ i.e., not in some adversarial simulation etc.
eyeundersand 7y ago
> A dragonfly nervous system is probably simpler than some of the largest artificial neural networks models. I would be extremely surprised if that were the case. Behavior of a single cell alone is unimaginably complex. Let alone an organ system, let alone the nervous system. I will agree with your Qualia point.

tartoran 7y ago

I am not an expert but from my understanding pain are signals perceived as what we subjectively call painful when something in our body/nervous system is damaged/inflamed or a system is out of whack. It is a very useful thing, it is trying to tell us something useful so we can act on it or just be aware of it. When something is out of whack permanently something in this system is malfunctioning. Without a body it is hard to describe the concept of pain and what it may feel to an emergent system with qualia properties. The closest I could think of is some kind of distress signal or base frequency change when the said system is under stress or wrong state and is perhaps used to self correct the system to a balanced state. If something is permanently painful then it ceases to be a reliable signal. Psychological pain or emotional pain or distress may come closer to what such a system could experience and but it could also be an emergent property of such a system but we don't fully understand how this works. If such a system has emotional capability then it should be able to experience it because pain is such a state of the emotional spectrum. No pain no joy - no oscillation/flat

totalZero 7y ago

We are all machines. The idea that one type of machine or another cannot feel "pain" is simply a rationalization for human behavior that causes harm to other organisms, like hunting and fishing and killing wasps. Pain is a sensation that drives an animal to quickly react to injurious situations. Consider a robot made with Brooks's subsumption architecture [1]. The basal level of behavior drive of the robot is its self-preservation. That level of behavior drive can halt all the higher-order behaviors to monopolize processing and locomotion. Imagine that this behavior drive is triggered when the ambient temperature approaches a level that can damage the battery, and attempts to move the robot to the lowest-temperature zone available. That is functionally no different than what happens to a person in a house fire. All we have done with the word "pain" is to set up an arbitrary delineation based on the hormonal/neuronal response that links our sensory apparatus to our behavioral drive. [1] https://people.csail.mit.edu/brooks/papers/representation.pdf https://people.csail.mit.edu/brooks/papers/representation.pd...

tartoran 7y ago
I agree with you, pain is a useful signal or it could be a systemic damage that wrongly sends these signals. However, when we, humans, refer to pain we clump all the unpleasant things we experience together. One such pain is the emotional /psychological pain and this type of pain is more likely to occur in such a system without a body. This is also a signal/state and is also useful to delineate it from another state. If such system is to experience joy it better know what pain is or otherwise wouldn't be able to delineate.
ben_w 7y ago
Careful, or you might use a definition of pain too broad to be useful. Plants, single-celled organisms, thermostats, and anti-lock braking systems meet your use of the word. Almost all of us know what it means to be in pain. It’s the hard to put into words subjective experience of suffering that is worth making ethical decisions about, not the autonomous reflex that doesn’t come with conciousness-is-a-poor-word-but-I-lack-a-better-alternative.
- totalZero 7y ago

russdill 7y ago

We're already really good at building mini-brains that can play pong..and much more advanced games than pong. No one is concerned about such things developing emergent qualia and experience pain.