3 ms·

Perhaps the right thing to do is place an AGI into a simulated world and observe its behaviour over (accelerated) eons, to understand its true nature. Perhaps t

by Gatsky 4y ago

Perhaps the right thing to do is place an AGI into a simulated world and observe its behaviour over (accelerated) eons, to understand its true nature. Perhaps that’s what our creator is doing to us.

melony 4y ago
Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!
- teraflop 4y ago
  For those who don't get the reference: A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment: > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1] and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective. [1]: https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html [2]: https://www.gregegan.net/MISC/CRYSTAL/Crystal.html https://www.gregegan.net/MISC/CRYSTAL/Crystal.html
  - fatherzine 4y ago
    Rather nihilistic perspective. There is a counterbalancing force, it's called "love", and our ancestors perhaps knew how to find it better than us, hyperstimulated beings enmeshed in an artificial environment. A couple of arbitrary references: https://www.vintag.es/2017/08/candid-shots-show-victorian-couple-from.html https://www.vintag.es/2017/08/candid-shots-show-victorian-co... https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot
  - Gatsky 4y ago
    Furthermore this is a significant ethical problem with testing and developing an AGI, especially an embodied one. That horrid montage of security footage in Ex Machina makes the point well.
  - djokkataja 4y ago
    > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. But by this logic, it's better not to have children, because they will inevitably experience some suffering, and if they have children, so will they, and so on. It's not that the logic is necessarily invalid from its own premises, but that it's self-extinguishing logic because entities that act based on it will stop propagating. There are also things that we can find worthwhile in living and creating new life. Not that that excuses any creation regardless of suffering, but it puts it in a gray zone where it's important to weigh the specificities of various suffering vs various joys as they come.
programmarchy 4y ago
This sounds like a wise thing to do, although to play devil’s advocate, if the AGI is hyper intelligent then it may be able to detect it’s inside a simulation and conceal any malicious intentions it may have.
- Gatsky 4y ago
  I think one could make a provably water tight simulation. Of course an AGI would realise it could be in a simulation, just as we have.
- fatherzine 4y ago
  Interesting thought. A small correction, from the AGI perspective, its intentions are pure and good. All beings are value function maximizers. The choice of the value function however... "To do evil a human being must first of all believe that what he's doing is good, or else that it's a well-considered act in conformity with natural law" -- Solzhenitsyn
  - tomcatfish 4y ago
    "The AI thinks it's good" Then why wouldn't it conceal it's good goal from those evil humans who would stop it from achieving it?
  - cookingrobot 4y ago
    Personally I don’t agree. I used to eat meat, then decided it was evil, then later stopped eating meat. There was a long time there where I kept doing it out of habit and laziness even though I thought it was wrong.

by Gatsky 4y ago

melony 4y ago

Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!

teraflop 4y ago
For those who don't get the reference: A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment: > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1] and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective. [1]: https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html [2]: https://www.gregegan.net/MISC/CRYSTAL/Crystal.html https://www.gregegan.net/MISC/CRYSTAL/Crystal.html
- fatherzine 4y ago
  Rather nihilistic perspective. There is a counterbalancing force, it's called "love", and our ancestors perhaps knew how to find it better than us, hyperstimulated beings enmeshed in an artificial environment. A couple of arbitrary references: https://www.vintag.es/2017/08/candid-shots-show-victorian-couple-from.html https://www.vintag.es/2017/08/candid-shots-show-victorian-co... https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot
- Gatsky 4y ago
  Furthermore this is a significant ethical problem with testing and developing an AGI, especially an embodied one. That horrid montage of security footage in Ex Machina makes the point well.
- djokkataja 4y ago
  > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. But by this logic, it's better not to have children, because they will inevitably experience some suffering, and if they have children, so will they, and so on. It's not that the logic is necessarily invalid from its own premises, but that it's self-extinguishing logic because entities that act based on it will stop propagating. There are also things that we can find worthwhile in living and creating new life. Not that that excuses any creation regardless of suffering, but it puts it in a gray zone where it's important to weigh the specificities of various suffering vs various joys as they come.

programmarchy 4y ago

This sounds like a wise thing to do, although to play devil’s advocate, if the AGI is hyper intelligent then it may be able to detect it’s inside a simulation and conceal any malicious intentions it may have.

Gatsky 4y ago
I think one could make a provably water tight simulation. Of course an AGI would realise it could be in a simulation, just as we have.
fatherzine 4y ago
Interesting thought. A small correction, from the AGI perspective, its intentions are pure and good. All beings are value function maximizers. The choice of the value function however... "To do evil a human being must first of all believe that what he's doing is good, or else that it's a well-considered act in conformity with natural law" -- Solzhenitsyn
- tomcatfish 4y ago
  "The AI thinks it's good" Then why wouldn't it conceal it's good goal from those evil humans who would stop it from achieving it?
- cookingrobot 4y ago
  Personally I don’t agree. I used to eat meat, then decided it was evil, then later stopped eating meat. There was a long time there where I kept doing it out of habit and laziness even though I thought it was wrong.