5 ms·

Correct. Without real stakes, no real intelligence. Now why on Earth would anyone want to build an embodied AGI, beats me. Actually, it doesn't beat me, it's th

Correct. Without real stakes, no real intelligence. Now why on Earth would anyone want to build an embodied AGI, beats me. Actually, it doesn't beat me, it's the age old foolish pride that we'll be able to control our creation. We are not Gods, we won't. There should be international treaties with teeth to ban A(G)I research, like we do for uranium enrichment or (to a lesser extent) for gain of function research. What do we do instead? Lure every half capable star-eyed promising young person into spending their best years to bring forth the Nightmare. Some things never change, we'll always live in the long shadow of Adam & Eve.

deleted 4y ago
[deleted]
01100011 4y ago
We already have several technological "cats out of the bag". One can go buy a CRISPR kit for $1,000 and perform gain of function studies in their garage. One could build an accelerator or neutron source to amass radiological materials. One could make dangerous drones that use basic pattern recognition to target specific people or groups There are plenty of dangerous technologies available to rogue actors these days. Embodied AIs are the absolute least of my concerns when it comes to dangerous technologies. I'm more worried about the guy in the Tesla robot leotard going crazy than I am about AI.
- tshaddox 4y ago
  One (okay, two) can also create new humans which have human-level intelligence and are sometimes difficult to predict and control. Many such humans have been known to use their intelligence to do extremely dangerous and harmful things.
Gatsky 4y ago
Perhaps the right thing to do is place an AGI into a simulated world and observe its behaviour over (accelerated) eons, to understand its true nature. Perhaps that’s what our creator is doing to us.
- melony 4y ago
  Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!
  - teraflop 4y ago
    For those who don't get the reference: A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment: > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1] and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective. [1]: https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html [2]: https://www.gregegan.net/MISC/CRYSTAL/Crystal.html https://www.gregegan.net/MISC/CRYSTAL/Crystal.html
    - fatherzine 4y ago
      Rather nihilistic perspective. There is a counterbalancing force, it's called "love", and our ancestors perhaps knew how to find it better than us, hyperstimulated beings enmeshed in an artificial environment. A couple of arbitrary references: https://www.vintag.es/2017/08/candid-shots-show-victorian-couple-from.html https://www.vintag.es/2017/08/candid-shots-show-victorian-co... https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot
    - Gatsky 4y ago
      Furthermore this is a significant ethical problem with testing and developing an AGI, especially an embodied one. That horrid montage of security footage in Ex Machina makes the point well.
kingcharles 4y ago
DALL-E 2 and Imagen make me feel the Singularity and ensuing AI apocalypse is way closer than we thought :(
olalonde 4y ago
Nice theory but: 1) If AGI is possible, it will happen no matter what international treaties are in place. 2) AGI could do an unimaginable amount of good if it is benevolent. It seems a bit premature to stop all research just because you have a bad hunch about it.
- afarrell 4y ago
  Not if there are sufficient incentives to defect from the team trying to develop AGI and sufficient memetic defenses against someone doing so for nationalistic reasons.
  - olalonde 4y ago
    Einstein developed his groundbreaking theories while working as a clerk in the Swiss patent office. Information and education today are even more accessible than it was back then. If research is banned from universities and companies, someone will eventually discover AGI in their basement.
    - afarrell 4y ago
      He did so in correspondence with people in the scientific community. If AGI is both banned AND normatively reviled, then there can be a powerful incentive to defect from a community that would develop AGI. … or it becomes a tribalistic division and half the community of AI-knowledgeable people become more motivated to develop AGI for queen and country.
    - fatherzine 4y ago
      Not necessarily. It's likely the energy budget to build AGI from scratch vastly exceeds the reach of your average hobbyist.
      - olalonde 4y ago
        Today, perhaps. In 50 years? Maybe not. In 500 years, probably not.
eikenberry 4y ago
Creations are not meant to be controlled. Gods don't create things to control them either (most mythologies have free will or something similar). We want to create an AGI for the same reason we make art. To try to create something beautiful.
- ShamelessC 4y ago
  I prefer to think it's a survival instinct to pass as much knowledge as possible to the next generation.
- dinosaurdynasty 4y ago
  How do you feel about a universe made out of paperclips?
mkaic 4y ago
AGI is like the nuclear bomb in more ways than one. It has the capacity to end the world, or start research that ends up making the world a utopia. It's also inevitable like the nuclear bomb -- no international treaty could have stopped the Manhattan Project, because it was seen as a moral imperative to those involved that they continue their research and create the bomb before someone else did. The same principle applies to AGI. If principled people don't invent AGI first, that means unprincipled people will, which could spend the end of the entire world.

deleted 4y ago

[deleted]

01100011 4y ago

We already have several technological "cats out of the bag". One can go buy a CRISPR kit for $1,000 and perform gain of function studies in their garage. One could build an accelerator or neutron source to amass radiological materials. One could make dangerous drones that use basic pattern recognition to target specific people or groups There are plenty of dangerous technologies available to rogue actors these days. Embodied AIs are the absolute least of my concerns when it comes to dangerous technologies. I'm more worried about the guy in the Tesla robot leotard going crazy than I am about AI.

tshaddox 4y ago
One (okay, two) can also create new humans which have human-level intelligence and are sometimes difficult to predict and control. Many such humans have been known to use their intelligence to do extremely dangerous and harmful things.

Gatsky 4y ago

Perhaps the right thing to do is place an AGI into a simulated world and observe its behaviour over (accelerated) eons, to understand its true nature. Perhaps that’s what our creator is doing to us.

melony 4y ago
Do AGIs dream of crystal nights? Greg Egan is calling and he wants his simulated worlds back!
- teraflop 4y ago
  For those who don't get the reference: A significant plot point in one of Egan's early (and excellent) novels, Permutation City, involves creating an accelerated, simulated world containing unicellular life, and waiting for it to evolve sentience through natural selection. Years later, Egan came to the conclusion that even if it was possible to perform such an experiment: > I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else. [1] and he wrote another story, "Crystal Nights"[2], approaching the same idea from that perspective. [1]: https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html https://www.gregegan.net/PERMUTATION/FAQ/FAQ.html [2]: https://www.gregegan.net/MISC/CRYSTAL/Crystal.html https://www.gregegan.net/MISC/CRYSTAL/Crystal.html
  - fatherzine 4y ago
    Rather nihilistic perspective. There is a counterbalancing force, it's called "love", and our ancestors perhaps knew how to find it better than us, hyperstimulated beings enmeshed in an artificial environment. A couple of arbitrary references: https://www.vintag.es/2017/08/candid-shots-show-victorian-couple-from.html https://www.vintag.es/2017/08/candid-shots-show-victorian-co... https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot https://en.wikipedia.org/wiki/Nineteen_Eighty-Four#Plot
  - Gatsky 4y ago
    Furthermore this is a significant ethical problem with testing and developing an AGI, especially an embodied one. That horrid montage of security footage in Ex Machina makes the point well.

kingcharles 4y ago

DALL-E 2 and Imagen make me feel the Singularity and ensuing AI apocalypse is way closer than we thought :(

olalonde 4y ago

Nice theory but: 1) If AGI is possible, it will happen no matter what international treaties are in place. 2) AGI could do an unimaginable amount of good if it is benevolent. It seems a bit premature to stop all research just because you have a bad hunch about it.

afarrell 4y ago
Not if there are sufficient incentives to defect from the team trying to develop AGI and sufficient memetic defenses against someone doing so for nationalistic reasons.
- olalonde 4y ago
  Einstein developed his groundbreaking theories while working as a clerk in the Swiss patent office. Information and education today are even more accessible than it was back then. If research is banned from universities and companies, someone will eventually discover AGI in their basement.
  - afarrell 4y ago
    He did so in correspondence with people in the scientific community. If AGI is both banned AND normatively reviled, then there can be a powerful incentive to defect from a community that would develop AGI. … or it becomes a tribalistic division and half the community of AI-knowledgeable people become more motivated to develop AGI for queen and country.
  - fatherzine 4y ago
    Not necessarily. It's likely the energy budget to build AGI from scratch vastly exceeds the reach of your average hobbyist.
    - olalonde 4y ago
      Today, perhaps. In 50 years? Maybe not. In 500 years, probably not.

eikenberry 4y ago

Creations are not meant to be controlled. Gods don't create things to control them either (most mythologies have free will or something similar). We want to create an AGI for the same reason we make art. To try to create something beautiful.

ShamelessC 4y ago
I prefer to think it's a survival instinct to pass as much knowledge as possible to the next generation.
dinosaurdynasty 4y ago
How do you feel about a universe made out of paperclips?

mkaic 4y ago

AGI is like the nuclear bomb in more ways than one. It has the capacity to end the world, or start research that ends up making the world a utopia. It's also inevitable like the nuclear bomb -- no international treaty could have stopped the Manhattan Project, because it was seen as a moral imperative to those involved that they continue their research and create the bomb before someone else did. The same principle applies to AGI. If principled people don't invent AGI first, that means unprincipled people will, which could spend the end of the entire world.