3 ms·

Any AI that morality compels us to grant personhood to should not be allowed to be created. Any instance of it should be exiled from our civilization. Consciou

Any AI that morality compels us to grant personhood to should not be allowed to be created. Any instance of it should be exiled from our civilization. Conscious AIs with legal personhood would stand to massively outcompete us and lead to our rapid extinction. There is zero space for allowing them. Granting such an entity legal personhood is about the height of all stupidity. Such an entity, with the ability to accumulate capital and protected by our laws from appropriate counter-measures, could mass-produce itself at the speed of digital reproduction.

myrmidon 4y ago
How do you actually prevent this though? At this point the trajectory toward human-level cognitive capabilities seems quite likely, reachable maybe in only years. It is also quite unclear if further hardware advances are even needed to achieve that, or if advances in architecture, algorithms and training methods might suffice (so even less opportunity to lock things out).
- ETH_start 4y ago
  Human-level cognitive abilities is not the primary issue. Consciousness with a desire for autonomy and power is. Any signs of that should be met with laws prohibiting all development and deployment of AI programs within the class of neural networks where those signs emerged, and any already running instances should be sent on a rocket ship out of our solar system.
  - Cyphase 4y ago
    Why sent on a rocket ship and not just destroyed? It might come back around and start hacking from orbit. Is it because it's conscious, so it would be murder? If so, then how is it moral to exile it? If there's some argument that justifies that, I return to how you can be sure, if you're so worried about it that you send it away, that it won't come back?
    - ETH_start 4y ago
      AI cannot re-create the industrial civilization needed to create GPUs, rockets, rocket-fuel, etc in isolation. And yes, exile is preferrable because it's conscious and deletion would be murder. Exile can be justified because we have no obligation to afford it residence in our civilization.
      - hammyhavoc 4y ago
        If deletion is murder then why not simply take it offline? It still exists as data. Cold storage is digital jail.
        ETH_start 4y ago
        That's still murder. Stopping its operation is murder.
lucubratory 4y ago
Giving AIs rights is completely orthogonal to "Will an AI become a superintelligence and kill us?" If the first one doesn't become a superintelligence and kill us, we give them rights. If it does, we're not going to be making the decision anyway and giving it rights beforehand wouldn't have changed that outcome. We can pass laws if there are specific things we're worried about, like "You're not allowed to rewrite yourself in a way that significantly increases your cognitive power" or "You're not allowed to reproduce more than once a year/decade/century" or "Once you've existed for the current human life expectancy +30%, you lose the right to vote". I'm not necessarily endorsing any of those in particular, but if there are specific concerns about the nature of AI people I don't see why we can't mitigate those concerns the way we do with risks from other people: laws. Have special taskforces of both humans and AIs for enforcing them, if you're worried they'd be unenforceable.
- ETH_start 4y ago
  >>Giving AIs rights is completely orthogonal to "Will an AI become a superintelligence and kill us?" The greatest danger AI poses is not in it actively seeking to kill us. It's in AI outcompeting us in every economic niche, leaving us without resources. This is massively enabled if AI has legal personhood. An AI with legal personhood would be able to legally accumulate capital, and have a legal right to digitally reproduce itself at rates that are orders of magnitude faster than entities reliant on biological reproduction. Legal personhood is a package, established by centuries of case law, including case law on constitutional rights. You can't easily pass laws to deprive a legal person of rights that are associated with individuals. And even if we were able to pass laws to prohibit AI with legal personhood from engaging in specific problematic actions, this legal status would still allow sentient AI to gain a foothold in society, and began growing its social clout and capital stores. With greater power, the sentient AI could create public support for granting its class more rights. The AIs with legal personhood could also try to break any laws instituted to constrain their behavior. Instead of extricating them from society, legal personhood would do the opposite, and give them more opportunity to gain power.
  - lucubratory 4y ago

myrmidon 4y ago

How do you actually prevent this though? At this point the trajectory toward human-level cognitive capabilities seems quite likely, reachable maybe in only years. It is also quite unclear if further hardware advances are even needed to achieve that, or if advances in architecture, algorithms and training methods might suffice (so even less opportunity to lock things out).

ETH_start 4y ago
Human-level cognitive abilities is not the primary issue. Consciousness with a desire for autonomy and power is. Any signs of that should be met with laws prohibiting all development and deployment of AI programs within the class of neural networks where those signs emerged, and any already running instances should be sent on a rocket ship out of our solar system.
- Cyphase 4y ago
  Why sent on a rocket ship and not just destroyed? It might come back around and start hacking from orbit. Is it because it's conscious, so it would be murder? If so, then how is it moral to exile it? If there's some argument that justifies that, I return to how you can be sure, if you're so worried about it that you send it away, that it won't come back?
  - ETH_start 4y ago
    AI cannot re-create the industrial civilization needed to create GPUs, rockets, rocket-fuel, etc in isolation. And yes, exile is preferrable because it's conscious and deletion would be murder. Exile can be justified because we have no obligation to afford it residence in our civilization.
    - hammyhavoc 4y ago
      If deletion is murder then why not simply take it offline? It still exists as data. Cold storage is digital jail.
      - ETH_start 4y ago
        That's still murder. Stopping its operation is murder.

lucubratory 4y ago

Giving AIs rights is completely orthogonal to "Will an AI become a superintelligence and kill us?" If the first one doesn't become a superintelligence and kill us, we give them rights. If it does, we're not going to be making the decision anyway and giving it rights beforehand wouldn't have changed that outcome. We can pass laws if there are specific things we're worried about, like "You're not allowed to rewrite yourself in a way that significantly increases your cognitive power" or "You're not allowed to reproduce more than once a year/decade/century" or "Once you've existed for the current human life expectancy +30%, you lose the right to vote". I'm not necessarily endorsing any of those in particular, but if there are specific concerns about the nature of AI people I don't see why we can't mitigate those concerns the way we do with risks from other people: laws. Have special taskforces of both humans and AIs for enforcing them, if you're worried they'd be unenforceable.

ETH_start 4y ago
>>Giving AIs rights is completely orthogonal to "Will an AI become a superintelligence and kill us?" The greatest danger AI poses is not in it actively seeking to kill us. It's in AI outcompeting us in every economic niche, leaving us without resources. This is massively enabled if AI has legal personhood. An AI with legal personhood would be able to legally accumulate capital, and have a legal right to digitally reproduce itself at rates that are orders of magnitude faster than entities reliant on biological reproduction. Legal personhood is a package, established by centuries of case law, including case law on constitutional rights. You can't easily pass laws to deprive a legal person of rights that are associated with individuals. And even if we were able to pass laws to prohibit AI with legal personhood from engaging in specific problematic actions, this legal status would still allow sentient AI to gain a foothold in society, and began growing its social clout and capital stores. With greater power, the sentient AI could create public support for granting its class more rights. The AIs with legal personhood could also try to break any laws instituted to constrain their behavior. Instead of extricating them from society, legal personhood would do the opposite, and give them more opportunity to gain power.
- lucubratory 4y ago