4 ms·

Maybe if the AI is impartial. But who trains the AI? Who guarantees it has no biases, implicit (because it's trained on how things work today, which is maybe no

by ale42 2y ago

Maybe if the AI is impartial. But who trains the AI? Who guarantees it has no biases, implicit (because it's trained on how things work today, which is maybe not what we want) or explicitly planted there (because of political plans of AI owners or other external influence)?

perching_aix 2y ago
Well, people keep riding the AGI train, no? Surely a superintelligence should be able to overcome its own biases, right? If there's any bar I'd pose for an intelligence to be "super", this would definitely be one of them. Even with regular humans, being open minded, willing to critically rethink their own stance, and being open for discussions is generally regarded as a sign of intelligence, after all.
- short_sells_poo 2y ago
  Your post might have been tongue in cheek, but assuming it's not: the issue with ceding decision making to a superintelligence is that we'll likely lose the ability to reason about it's decisions and we'll struggle to take back control if it starts making choices we don't like. It's reasoning is far beyond our own capabilities after all.
  - perching_aix 2y ago
    It's half tongue-in-cheek. That being said, the larger picture behind the decisionmaking of our human leaders is also essentially uncheckable, and is often a key part of many conspiracy theories as a result.
- ale42 2y ago
  Many (maybe most) humans are not even aware of their own biases, though they are supposed to have "general" intelligence. When they are aware, sometimes they use various psychological strategies to live with them or justify them instead of trying to overcome them. So the AGI in question must be "better" than most humans in this sense. Plus, there's the question about who would control that AGI. If it's a black box in the hands of a company, how can I know for sure that the AGI has no secret plan implanted by the company or by some government?

by ale42 2y ago

perching_aix 2y ago

Well, people keep riding the AGI train, no? Surely a superintelligence should be able to overcome its own biases, right? If there's any bar I'd pose for an intelligence to be "super", this would definitely be one of them. Even with regular humans, being open minded, willing to critically rethink their own stance, and being open for discussions is generally regarded as a sign of intelligence, after all.

short_sells_poo 2y ago
Your post might have been tongue in cheek, but assuming it's not: the issue with ceding decision making to a superintelligence is that we'll likely lose the ability to reason about it's decisions and we'll struggle to take back control if it starts making choices we don't like. It's reasoning is far beyond our own capabilities after all.
- perching_aix 2y ago
  It's half tongue-in-cheek. That being said, the larger picture behind the decisionmaking of our human leaders is also essentially uncheckable, and is often a key part of many conspiracy theories as a result.
ale42 2y ago
Many (maybe most) humans are not even aware of their own biases, though they are supposed to have "general" intelligence. When they are aware, sometimes they use various psychological strategies to live with them or justify them instead of trying to overcome them. So the AGI in question must be "better" than most humans in this sense. Plus, there's the question about who would control that AGI. If it's a black box in the hands of a company, how can I know for sure that the AGI has no secret plan implanted by the company or by some government?