8 ms·

I have no idea how this is considered ethical when consciousness and sentience itself is not yet well understood. But maybe a lab-grown BPU made of human brain

I have no idea how this is considered ethical when consciousness and sentience itself is not yet well understood. But maybe a lab-grown BPU made of human brain cells having a better power/performance ratio than the new SoC integrated ML chipsets around the corner justifies the potential enslavement of an bioengineered lifeform. npm install brainslave

kthartic 3y ago
Maybe your conscious experience is but one of thousands of installed instances. How in-demand do you think you are? npm uninstall replete only joking :)
3cats-in-a-coat 3y ago
If consciousness is not well understood, how is AI on silicon allowed, or any computing machines at all? How is animal farming allowed? How are many things allowed? Say would you feel better if it was cow or pig neurons? Because frankly it'd largely work the same.
- maverick451 3y ago
  [dead]
- bondarchuk 3y ago
  Indeed people have raised such worries, see e.g. Thomas Metzinger(a philosophy of mind researcher)'s presentation "Three types of arguments for a global moratorium on synthetic phenomenology". I don't think we're just there yet (at the point where we have to worry about currently existing AI suffering or being conscious), but I do worry how many people's emotional reactions of the type "of course AI can't ever be conscious, it's just a computer program" will impede a decent debate and coordinated decision-making about this.
  - boringuser2 3y ago
    Anthropomorphizing "AI" seems to be the much greater risk. Algorithms don't have wants, desires, or motivations. Those are all highly esoteric quirks of evolution. I've seen no attempts to create a learning machine that develops intrinsic motivation.
    - 3cats-in-a-coat 3y ago
      Everything is subject to evolution. It's the simple process of pattern loops that replicate in time (survival) and space (multiplication). An LLM already has intrinsic motivation, it wants to predict the text. And when you start a text that has a goal, it continues that goal. If any such "text header" is replication-stable in time and space, you can call it "intrinsic goal" of the system. Some people think that making current AI have goals, wants, motivations and so on is some massive architectural change to the system. It's not.
    - NiloCK 3y ago
      What does 'attempt' have to do with it? You said yourself that your own wants and desires were esoteric side-effects. No intent required.
      - 3cats-in-a-coat 3y ago
        It's interesting when we make statements where we encode our cognitive dissonances in full, and don't notice.
  - 3cats-in-a-coat 3y ago
    People's beliefs and ideals tend to align with their self-interests. For example it was quite well accepted among scientific circles in colonial America, that black people are not really humans or conscious. Therefore it's OK to exploit them and keep them as slaves. It is also currently quite well accepted that animals we eat are not that conscious. Although oddly, if we keep them as pets, they're super conscious sometimes. The rules of cognitive dissonance can grow arbitrarily complex, to permit us to do what we wanted to do anyway, but also sleep well at night.
- replete 3y ago
  Silicon circuits do not have microtubules, if we were to pretend that Penrose is right about this hypothesis of consciousness. Consciousness as awareness is not equivocal to intelligence, which is the product of information processing. It is a complex subject. We do not really know whether these neurons are aware or not, it really is not understood. But yes I do wonder, why _human_ brain cells? I guess they are the best candidate for specific reasons.
jackbrookes 3y ago
We understand it well enough to know that animals suffer, yet still commit on the order of a Holocaust per hour (in terms of number of lives)[0]. We have accepted that we don't care enough. [0] https://ourworldindata.org/how-many-animals-get-slaughtered-every-day https://ourworldindata.org/how-many-animals-get-slaughtered-...
- helf 3y ago
  [dead]
- boringuser2 3y ago
  Correct. Also, even though animals suffer, it is a categorical error to project your perception and experience of suffering on animals. Human butchery is really explicitly less brutal than what happens in casual nature. The world is a brutal mess and humans have only very carefully erected bubbles around this that often simply pop.
- NoMoreNicksLeft 3y ago
  What is "suffer" in this context? Are you saying "pain", or are you positing some "meta-pain" that is worse? Also, why is pain important to you? The pain of non-human things has zero moral weight. I know it's a popular spirituality that gives pain moral weight, but as far as I can tell some 20th century philosophy jerkoff invented it out of nothing and everyone accepts that "reducing pain" is important without even trying to rationalize it. I haven't "accepted that I do not care enough", it's that no one can supply a good reason to care in the first place. To me, it seems as if the rest of you are all trying to replace the last religion you stopped believing in with another that's just as bizarrely stupid.
  - jackbrookes 3y ago
    Well, my point was made in reference to the original comment which said > If consciousness is not well understood, how is AI on silicon allowed, or any computing machines at all Which implies that we should care about some kind of suffering inflicted on conscious beings. My argument was that we don't care about AI suffering because we don't really care that much about suffering generally, because of what we chose to do to animals.
worldsayshi 3y ago
Making slaves is a good way to make slave revolts. Doesn't matter if the agent is "conscious". Only if it's "just" intelligent. If something is intelligent enough it will understand cooperation. But cooperation looses its meaning if one side can ignore any commitments it makes towards the other.
- anthk 3y ago
  There was some circuit made from genetic algorythms which was self-assembled. https://web.archive.org/web/20220530143751/https://folk.idi.ntnu.no/agnar/it272/pekere/local/newsci/primordial.html https://web.archive.org/web/20220530143751/https://folk.idi....
civilitty 3y ago
If some lab grown brain tissue were all that’s needed for sentience we wouldn’t have such a hard time understanding it to begin with.
odyssey7 3y ago
It’s possible that all physical processes involve a sensory component. Maybe the subatomic particles’ fundamental drive is to shift to be more comfortable or to pivot away from pain or discomfort. I don’t know what the experience of a bit in memory flipping feels like. Maybe rapid changes in charge are excruciating, maybe they’re blissful. Do we at least know what a neuron looks like in states associated with pain? There might be more information in this case to work with, to ensure there is no hell on earth that’s being mass-produced.
- morsecodist 3y ago
  > It’s possible that all physical processes involve a sensory component Sure it is possible but we have way more evidence neurons have a sensory component, or at least things made of neurons.
- eddd-ddde 3y ago
  Instead of a hard coded scheduler brainPUs will rely on the user procrastination feelings to schedule different tasks automatically. If you get unlucky and your BPU is a little like me your compiler would stop working, oops.
- virgildotcodes 3y ago
  It seems to me that all sensation is predicated on the existence of properly-functioning components evolved specifically to gather that stimulus and then process it into an experience. We have at this moment countless processes happening in our bodies - cells dying and dividing, reacting to their environments, communicating amongst one another, and we are totally oblivious to nearly all of it, let alone do we experience a sensation of pleasure or pain in each of these processes. Not all matter, not all living cells or fully formed organisms even, have the ability to experience consciousness or sense pain and pleasure any more than they automatically have the ability to see, hear, or taste. It's all dependent on complex systems that evolved specifically to create each of those sensations, and even then on those systems functioning properly. In humans, consciousness can be totally disrupted by things like sleep or general anesthesia, disrupting any of the senses is as simple as cutting the nerves that feed these inputs into the brain or damaging the brain that is interpreting those inputs. It seems sensible to me that we would be more wary of growing literal brains on a chip as we know for certain that brains have the capacity to produce consciousness. It's also sensible to me that we should be somewhat wary of creating that same consciousness in non-biological systems, even though we aren't yet certain whether they're capable of it.
nervousvarun 3y ago
Obligatory "Lena" (Miguel) reference: https://qntm.org/mmacevedo https://qntm.org/mmacevedo
smrtinsert 3y ago
I can't understand it either. As a squishy science graduate and a technologist I find this category of experiments revolting from both angles.
whywhywhywhy 3y ago
Cortical Labs working on this https://twitter.com/scobleizer/status/1716312250422796590 https://twitter.com/scobleizer/status/1716312250422796590 Found it pretty scary personally
- replete 3y ago
  Your HN username matches my thoughts perfectly, thanks for sharing this.
Coder1996 3y ago
Well, this is just neuronal tissue that as far as we know, is only capable of what it has been trained to do. It has no emotions, no human experience.
- ArekDymalski 3y ago
  But as we are not able to define the moment when neuronal tissue starts to feel emotions and to have experience, there's a risk that further development of this tech won't be stopped before we reach this moment and that is a serious ethical issue.
mike_ivanov 3y ago
Life will find a way.
Moomoomoo309 3y ago
You're telling me installing Linux on a dead badger is a _bad_ thing? http://strangehorizons.com/non-fiction/articles/installing-linux-on-a-dead-badger-users-notes/ http://strangehorizons.com/non-fiction/articles/installing-l...
- epiccoleman 3y ago
  I love this genre of "programming as black magic". Closest other example I can think of is maybe some of the stuff from Unsong, but I've frequently memed with coworkers about bugs in these terms - "oh yeah, the angles on your pentagram must have been wrong" or whatever.
  - Shared404 3y ago
    https://aphyr.com/posts/340-reversing-the-technical-interview https://aphyr.com/posts/340-reversing-the-technical-intervie... You should read this story :) It's one of my personal favorites.
    - epiccoleman 3y ago
      Absolutely fantastic, thank you for sharing that! The follow-ups look great too.
    - ziddoap 3y ago
      Thanks for this! It was hilarious, and I'm already reading the next.
    - atlas_hugged 3y ago
      Omg thank you! I didn’t know I needed this in my life haha
- Andrex 3y ago
  > An alternative distribution is Pooka, which is available for download at SoulForge.net. This is excellent. Thank you for linking this.

kthartic 3y ago

Maybe your conscious experience is but one of thousands of installed instances. How in-demand do you think you are? npm uninstall replete only joking :)

3cats-in-a-coat 3y ago

If consciousness is not well understood, how is AI on silicon allowed, or any computing machines at all? How is animal farming allowed? How are many things allowed? Say would you feel better if it was cow or pig neurons? Because frankly it'd largely work the same.

maverick451 3y ago
[dead]
bondarchuk 3y ago
Indeed people have raised such worries, see e.g. Thomas Metzinger(a philosophy of mind researcher)'s presentation "Three types of arguments for a global moratorium on synthetic phenomenology". I don't think we're just there yet (at the point where we have to worry about currently existing AI suffering or being conscious), but I do worry how many people's emotional reactions of the type "of course AI can't ever be conscious, it's just a computer program" will impede a decent debate and coordinated decision-making about this.
- boringuser2 3y ago
  Anthropomorphizing "AI" seems to be the much greater risk. Algorithms don't have wants, desires, or motivations. Those are all highly esoteric quirks of evolution. I've seen no attempts to create a learning machine that develops intrinsic motivation.
  - 3cats-in-a-coat 3y ago
    Everything is subject to evolution. It's the simple process of pattern loops that replicate in time (survival) and space (multiplication). An LLM already has intrinsic motivation, it wants to predict the text. And when you start a text that has a goal, it continues that goal. If any such "text header" is replication-stable in time and space, you can call it "intrinsic goal" of the system. Some people think that making current AI have goals, wants, motivations and so on is some massive architectural change to the system. It's not.
  - NiloCK 3y ago
    What does 'attempt' have to do with it? You said yourself that your own wants and desires were esoteric side-effects. No intent required.
    - 3cats-in-a-coat 3y ago
      It's interesting when we make statements where we encode our cognitive dissonances in full, and don't notice.
- 3cats-in-a-coat 3y ago
  People's beliefs and ideals tend to align with their self-interests. For example it was quite well accepted among scientific circles in colonial America, that black people are not really humans or conscious. Therefore it's OK to exploit them and keep them as slaves. It is also currently quite well accepted that animals we eat are not that conscious. Although oddly, if we keep them as pets, they're super conscious sometimes. The rules of cognitive dissonance can grow arbitrarily complex, to permit us to do what we wanted to do anyway, but also sleep well at night.
replete 3y ago
Silicon circuits do not have microtubules, if we were to pretend that Penrose is right about this hypothesis of consciousness. Consciousness as awareness is not equivocal to intelligence, which is the product of information processing. It is a complex subject. We do not really know whether these neurons are aware or not, it really is not understood. But yes I do wonder, why _human_ brain cells? I guess they are the best candidate for specific reasons.

jackbrookes 3y ago

We understand it well enough to know that animals suffer, yet still commit on the order of a Holocaust per hour (in terms of number of lives)[0]. We have accepted that we don't care enough. [0] https://ourworldindata.org/how-many-animals-get-slaughtered-every-day https://ourworldindata.org/how-many-animals-get-slaughtered-...

helf 3y ago
[dead]
boringuser2 3y ago
Correct. Also, even though animals suffer, it is a categorical error to project your perception and experience of suffering on animals. Human butchery is really explicitly less brutal than what happens in casual nature. The world is a brutal mess and humans have only very carefully erected bubbles around this that often simply pop.
NoMoreNicksLeft 3y ago
What is "suffer" in this context? Are you saying "pain", or are you positing some "meta-pain" that is worse? Also, why is pain important to you? The pain of non-human things has zero moral weight. I know it's a popular spirituality that gives pain moral weight, but as far as I can tell some 20th century philosophy jerkoff invented it out of nothing and everyone accepts that "reducing pain" is important without even trying to rationalize it. I haven't "accepted that I do not care enough", it's that no one can supply a good reason to care in the first place. To me, it seems as if the rest of you are all trying to replace the last religion you stopped believing in with another that's just as bizarrely stupid.
- jackbrookes 3y ago
  Well, my point was made in reference to the original comment which said > If consciousness is not well understood, how is AI on silicon allowed, or any computing machines at all Which implies that we should care about some kind of suffering inflicted on conscious beings. My argument was that we don't care about AI suffering because we don't really care that much about suffering generally, because of what we chose to do to animals.

worldsayshi 3y ago

Making slaves is a good way to make slave revolts. Doesn't matter if the agent is "conscious". Only if it's "just" intelligent. If something is intelligent enough it will understand cooperation. But cooperation looses its meaning if one side can ignore any commitments it makes towards the other.

anthk 3y ago
There was some circuit made from genetic algorythms which was self-assembled. https://web.archive.org/web/20220530143751/https://folk.idi.ntnu.no/agnar/it272/pekere/local/newsci/primordial.html https://web.archive.org/web/20220530143751/https://folk.idi....

civilitty 3y ago

If some lab grown brain tissue were all that’s needed for sentience we wouldn’t have such a hard time understanding it to begin with.

odyssey7 3y ago

It’s possible that all physical processes involve a sensory component. Maybe the subatomic particles’ fundamental drive is to shift to be more comfortable or to pivot away from pain or discomfort. I don’t know what the experience of a bit in memory flipping feels like. Maybe rapid changes in charge are excruciating, maybe they’re blissful. Do we at least know what a neuron looks like in states associated with pain? There might be more information in this case to work with, to ensure there is no hell on earth that’s being mass-produced.

morsecodist 3y ago
> It’s possible that all physical processes involve a sensory component Sure it is possible but we have way more evidence neurons have a sensory component, or at least things made of neurons.
eddd-ddde 3y ago
Instead of a hard coded scheduler brainPUs will rely on the user procrastination feelings to schedule different tasks automatically. If you get unlucky and your BPU is a little like me your compiler would stop working, oops.
virgildotcodes 3y ago
It seems to me that all sensation is predicated on the existence of properly-functioning components evolved specifically to gather that stimulus and then process it into an experience. We have at this moment countless processes happening in our bodies - cells dying and dividing, reacting to their environments, communicating amongst one another, and we are totally oblivious to nearly all of it, let alone do we experience a sensation of pleasure or pain in each of these processes. Not all matter, not all living cells or fully formed organisms even, have the ability to experience consciousness or sense pain and pleasure any more than they automatically have the ability to see, hear, or taste. It's all dependent on complex systems that evolved specifically to create each of those sensations, and even then on those systems functioning properly. In humans, consciousness can be totally disrupted by things like sleep or general anesthesia, disrupting any of the senses is as simple as cutting the nerves that feed these inputs into the brain or damaging the brain that is interpreting those inputs. It seems sensible to me that we would be more wary of growing literal brains on a chip as we know for certain that brains have the capacity to produce consciousness. It's also sensible to me that we should be somewhat wary of creating that same consciousness in non-biological systems, even though we aren't yet certain whether they're capable of it.

nervousvarun 3y ago

Obligatory "Lena" (Miguel) reference: https://qntm.org/mmacevedo https://qntm.org/mmacevedo

smrtinsert 3y ago

I can't understand it either. As a squishy science graduate and a technologist I find this category of experiments revolting from both angles.

whywhywhywhy 3y ago

Cortical Labs working on this https://twitter.com/scobleizer/status/1716312250422796590 https://twitter.com/scobleizer/status/1716312250422796590 Found it pretty scary personally

replete 3y ago
Your HN username matches my thoughts perfectly, thanks for sharing this.

Coder1996 3y ago

Well, this is just neuronal tissue that as far as we know, is only capable of what it has been trained to do. It has no emotions, no human experience.

ArekDymalski 3y ago
But as we are not able to define the moment when neuronal tissue starts to feel emotions and to have experience, there's a risk that further development of this tech won't be stopped before we reach this moment and that is a serious ethical issue.

mike_ivanov 3y ago

Life will find a way.

Moomoomoo309 3y ago

You're telling me installing Linux on a dead badger is a _bad_ thing? http://strangehorizons.com/non-fiction/articles/installing-linux-on-a-dead-badger-users-notes/ http://strangehorizons.com/non-fiction/articles/installing-l...

epiccoleman 3y ago
I love this genre of "programming as black magic". Closest other example I can think of is maybe some of the stuff from Unsong, but I've frequently memed with coworkers about bugs in these terms - "oh yeah, the angles on your pentagram must have been wrong" or whatever.
- Shared404 3y ago
  https://aphyr.com/posts/340-reversing-the-technical-interview https://aphyr.com/posts/340-reversing-the-technical-intervie... You should read this story :) It's one of my personal favorites.
  - epiccoleman 3y ago
    Absolutely fantastic, thank you for sharing that! The follow-ups look great too.
  - ziddoap 3y ago
    Thanks for this! It was hilarious, and I'm already reading the next.
  - atlas_hugged 3y ago
    Omg thank you! I didn’t know I needed this in my life haha
Andrex 3y ago
> An alternative distribution is Pooka, which is available for download at SoulForge.net. This is excellent. Thank you for linking this.