4 ms·
> almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats. Yes, exactly. Human organizations can be
by jjoonathan 3y ago
> almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
Yes, exactly. Human organizations can be terrifyingly powerful and awful even with human limitations. Human organizations are guaranteed to have inefficiency from low-bandwidth communication and a million principle-agent-problem fracture points created by the need for delegation, not to mention that power ultimately has to be centralized in slow, squishy, easy-to-kill bodies. AI organizations are not subject to any those limitations. Starting with a terrifying power and removing a bunch of limitations could lead to a very bad place.
> The only realistic path to a global AI threat is a subset of the "nuclear war"
No, it can just grow a traditional organization until it's Too Big to Turn Off.
- pjc50 3y ago> No, it can just grow a traditional organization until it's Too Big to Turn Off. Yeah, this is what I meant by "AI alignment" being inseparable from "corporate alignment".
- jjoonathan 3y ago...which we are notoriously bad at. We don't disagree on the analogy, we disagree on the analogy being a hopeful one.