4 ms·

> If AI takes over the world, This is a potential concern, but no currently-existing computer system (that I'm aware of) is any kind of risk for that. Certainl

> If AI takes over the world, This is a potential concern, but no currently-existing computer system (that I'm aware of) is any kind of risk for that. Certainly, nothing OpenAI has produced is. I'd say the currently-real unaligned-agent behaviour of our institutions is a bigger risk than currently-fictional computer-instantiated unaligned superintelligence. And we do have regulation in place to constrain the behaviour of many institutions: governments, corporations, utility operators, and so on. The unaligned agent problem was not discovered by the AI safety people. They merely named it.

Gormo 3y ago
> The unaligned agent problem was not discovered by the AI safety people. They merely named it. In other disciplines, this is known as the "principal-agent problem" [1] or "fiduciary risk" [2]. Like you, I'm far more worried about the behavior of human institutions controlled by third parties with their own agendas than I am about the behavior of software that implements stochastic algorithms. The risk in both cases comes from people surrendering their own agency and autonomy to external parties, and we already live in a word where too many people put blind faith in dysfunctional institutions which are demonstrably "unaligned" with the actual values and interests of those they pretend to act on behalf of. Turning to those very institutions to stop people from relying on "unaligned" software seems absolutely insane. ---- [1]: https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble... [2]: https://en.wikipedia.org/wiki/Fiduciary#Accountability https://en.wikipedia.org/wiki/Fiduciary#Accountability