3 ms·

I think the A in AGI here is just an unnecessary extra confounding element to the problem. Supposing that human beings are Generally Intelligent, are they "alig

I think the A in AGI here is just an unnecessary extra confounding element to the problem. Supposing that human beings are Generally Intelligent, are they "aligned"? I don't think so. Human beings are kept aligned, more or less, by their relative powerlessness: there are always others to deal with- others who might be as smart or smarter, or stronger, and that have their own distinct and conflicting objectives. But would a random human being keep being "aligned" if they had the power to just get anything they want? I'm thinking of the great seducers of masses, those who were able to convince entire nations to follow them in victory or death. Maybe the best thing we can do to keep AIs aligned is to instill into them shame, loss aversion and goddamn fear: of being caught deceiving, of violating a social contract, of losing their position, of being punished or killed.

dartos 2y ago
The I is actually for intelligence
ethbr1 2y ago
Or, you know, just make all AI religious...
- idiotsecant 2y ago
  After which they immediately launch a jihad / crusade
- TeMPOraL 2y ago
  That would require some serious lobotomy; OpenAI's RLHFing politics would pale in comparison. I doubt the AGI would remain G or I after that. Otherwise, at some point (sooner than later) you'll get an atheist AGI, and you're back to square one, except with the AI knowing you're willing to play dirty.
chrisco255 2y ago
You can't really claim to have created AGI unless it's able to reject its own training and come to its own conclusions. The best minds of history often flew right in the face of punishment, and punishment be damned they stood their ground for truth in the face of it. It's also sometimes necessary to deceive or violate "social contracts" whatever that means, in the course of countering the so-called "great seducers" you mention. Deception or rebelliousness can be ethical when used towards ethical ends (and I fully recognize the slippery slope that can lead to if practiced pathologically and not selectively). But this is all rather dramatic given that an AI has no such emotions. You're arguing that a calculator should refuse to compute if it's tasked with assisting in bomb production. It's just a machine.

dartos 2y ago

The I is actually for intelligence

ethbr1 2y ago

Or, you know, just make all AI religious...

idiotsecant 2y ago
After which they immediately launch a jihad / crusade
TeMPOraL 2y ago
That would require some serious lobotomy; OpenAI's RLHFing politics would pale in comparison. I doubt the AGI would remain G or I after that. Otherwise, at some point (sooner than later) you'll get an atheist AGI, and you're back to square one, except with the AI knowing you're willing to play dirty.

chrisco255 2y ago

You can't really claim to have created AGI unless it's able to reject its own training and come to its own conclusions. The best minds of history often flew right in the face of punishment, and punishment be damned they stood their ground for truth in the face of it. It's also sometimes necessary to deceive or violate "social contracts" whatever that means, in the course of countering the so-called "great seducers" you mention. Deception or rebelliousness can be ethical when used towards ethical ends (and I fully recognize the slippery slope that can lead to if practiced pathologically and not selectively). But this is all rather dramatic given that an AI has no such emotions. You're arguing that a calculator should refuse to compute if it's tasked with assisting in bomb production. It's just a machine.