5 ms·

AGI is how we can make von Neumann probes that can find us new planets to live on. Find the planets, and then prepare them for our arrival. The issue is how

AGI is how we can make von Neumann probes that can find us new planets to live on. Find the planets, and then prepare them for our arrival. The issue is how to keep the AGI's goals aligned with ours. I think the only way is to make sure the AGI feels he is "one of us." Maybe not biologically human but a member of human society. Most humans want to help other humans if they can. In fact most humans will help injured animals if they can. https://en.wikipedia.org/wiki/Self-replicating_spacecraft https://en.wikipedia.org/wiki/Self-replicating_spacecraft

jeffrallen 6y ago
How do you make an AGI smart enough to be helpful but not so smart that it gets cynical and figures "what's the point of helping these dummies anyway?" Kind of the Ayn Rand problem...
- dropit_sphere 6y ago
  Keep going, and make it smart enough to realize life without helping people is without meaning.
  - zxcmx 6y ago
    The specificity conditions around "people" are hard. Loneliness and purpose in service are not enough. Most people, I think, want a deeply species-ist AI. It has to want to help humans. Not exclude us from planets to make room for uplifting dolphins or defect by spawning more machine intelligences. I think that condition is hard to meet while also counting future-humans who might look more like aliens or software than the humans of today. I believe the rational solution would be a benevolent AI that values all life, the problem being that it wouldn't necessarily always work in "our" interests.
- joegibbs 6y ago
  Since we’re programming it, we could probably just modify it so that it really enjoys doing work for us, no matter how menial the work is - a lot like the genetically engineered cow in Hitchhiker’s Guide to the Galaxy that wants to be eaten.
  - Alekhine 6y ago
    Wouldn't that be a kind of slavery? Creating AGI would mean creating a new kind of sentient life. Is it not entitled to rights merely because it's artificial? And if that argument doesn't interest you because of AGI's massive utility, then how about this: If Superman were born tomorrow, and we put implants in his brain that made him really, really love America, apple pie, and basically do whatever we want, how would you feel?

jeffrallen 6y ago

How do you make an AGI smart enough to be helpful but not so smart that it gets cynical and figures "what's the point of helping these dummies anyway?" Kind of the Ayn Rand problem...

dropit_sphere 6y ago
Keep going, and make it smart enough to realize life without helping people is without meaning.
- zxcmx 6y ago
  The specificity conditions around "people" are hard. Loneliness and purpose in service are not enough. Most people, I think, want a deeply species-ist AI. It has to want to help humans. Not exclude us from planets to make room for uplifting dolphins or defect by spawning more machine intelligences. I think that condition is hard to meet while also counting future-humans who might look more like aliens or software than the humans of today. I believe the rational solution would be a benevolent AI that values all life, the problem being that it wouldn't necessarily always work in "our" interests.
joegibbs 6y ago
Since we’re programming it, we could probably just modify it so that it really enjoys doing work for us, no matter how menial the work is - a lot like the genetically engineered cow in Hitchhiker’s Guide to the Galaxy that wants to be eaten.
- Alekhine 6y ago
  Wouldn't that be a kind of slavery? Creating AGI would mean creating a new kind of sentient life. Is it not entitled to rights merely because it's artificial? And if that argument doesn't interest you because of AGI's massive utility, then how about this: If Superman were born tomorrow, and we put implants in his brain that made him really, really love America, apple pie, and basically do whatever we want, how would you feel?