3 ms·

Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!

by melony 4y ago

teraflop 4y ago
For those who don't get the reference: A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment: > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1] and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective. [1]: https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html [2]: https://www.gregegan.net/MISC/CRYSTAL/Crystal.html https://www.gregegan.net/MISC/CRYSTAL/Crystal.html
- fatherzine 4y ago
  Rather nihilistic perspective. There is a counterbalancing force, it's called "love", and our ancestors perhaps knew how to find it better than us, hyperstimulated beings enmeshed in an artificial environment. A couple of arbitrary references: https://www.vintag.es/2017/08/candid-shots-show-victorian-couple-from.html https://www.vintag.es/2017/08/candid-shots-show-victorian-co... https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot
- Gatsky 4y ago
  Furthermore this is a significant ethical problem with testing and developing an AGI, especially an embodied one. That horrid montage of security footage in Ex Machina makes the point well.
- djokkataja 4y ago
  > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. But by this logic, it's better not to have children, because they will inevitably experience some suffering, and if they have children, so will they, and so on. It's not that the logic is necessarily invalid from its own premises, but that it's self-extinguishing logic because entities that act based on it will stop propagating. There are also things that we can find worthwhile in living and creating new life. Not that that excuses any creation regardless of suffering, but it puts it in a gray zone where it's important to weigh the specificities of various suffering vs various joys as they come.

Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!

by melony 4y ago

Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!

For those who don't get the reference: A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment: > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1] and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective. [1]: https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html [2]: https://www.gregegan.net/MISC/CRYSTAL/Crystal.html https://www.gregegan.net/MISC/CRYSTAL/Crystal.html

fatherzine 4y ago
Rather nihilistic perspective. There is a counterbalancing force, it's called "love", and our ancestors perhaps knew how to find it better than us, hyperstimulated beings enmeshed in an artificial environment. A couple of arbitrary references: https://www.vintag.es/2017/08/candid-shots-show-victorian-couple-from.html https://www.vintag.es/2017/08/candid-shots-show-victorian-co... https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot
Gatsky 4y ago
Furthermore this is a significant ethical problem with testing and developing an AGI, especially an embodied one. That horrid montage of security footage in Ex Machina makes the point well.
djokkataja 4y ago
> I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. But by this logic, it's better not to have children, because they will inevitably experience some suffering, and if they have children, so will they, and so on. It's not that the logic is necessarily invalid from its own premises, but that it's self-extinguishing logic because entities that act based on it will stop propagating. There are also things that we can find worthwhile in living and creating new life. Not that that excuses any creation regardless of suffering, but it puts it in a gray zone where it's important to weigh the specificities of various suffering vs various joys as they come.