4 ms·

So like, a sociopath, then? I guess we do know that those can exist.

• TeMPOraL 7y ago
  Yeah, sure. The orthogonality thesis essentially implies than a GAI randomly plucked out of space of possible minds will likely be considered sociopathic by our standards. That is, if we can comprehend its thinking at all. "Not sociopath" is a very particular set of values.
  • ForHackernews 7y ago
    > a GAI randomly plucked out of space of possible minds Sorry, but I don't think you have any rational basis for imagining what the "space of possible minds" represents. The only minds with human-level intelligence we know of are human minds that have (with variation) human values. The claim you're making is analogous to saying "any extraterrestrial life we find won't be carbon-based because out of the space of all possible substrates for life, carbon is a very particular one", but that's an ill-founded supposition because we have a sample of N=1 and maybe carbon-based life is the only kind of life there is. Maybe a true GAI mind will be "like us", maybe it won't be, but we don't have anywhere near enough data to speak with confidence about it.