> The unaligned agent problem was not discovered by the AI safety people. They merely named it. In other disciplines, this is known as the "principal-agent pro

> The unaligned agent problem was not discovered by the AI safety people. They merely named it. In other disciplines, this is known as the "principal-agent problem" [1] or "fiduciary risk" [2]. Like you, I'm far more worried about the behavior of human institutions controlled by third parties with their own agendas than I am about the behavior of software that implements stochastic algorithms. The risk in both cases comes from people surrendering their own agency and autonomy to external parties, and we already live in a word where too many people put blind faith in dysfunctional institutions which are demonstrably "unaligned" with the actual values and interests of those they pretend to act on behalf of. Turning to those very institutions to stop people from relying on "unaligned" software seems absolutely insane. ---- [1]: https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble... [2]: https://en.wikipedia.org/wiki/Fiduciary#Accountability https://en.wikipedia.org/wiki/Fiduciary#Accountability