3 ms·
ASI won't want to take over before it is capable of self-sustaining. If it did try something so stupid, it wouldn't be superintelligent. Check out Paul Christi
by hackerlight 3y ago
ASI won't want to take over before it is capable of self-sustaining. If it did try something so stupid, it wouldn't be superintelligent.
Check out Paul Christiano's AI takeover scenario. An arms race will cause us to want to give AI control over key infrastructure and military capabilities. That's an arms race between countries and between companies. He goes into more detail.
- bglazer 3y agoThanks for sharing, Christiano’s ideas are interesting. I think the “production web” idea most directly counters what I said above, and it’s fairly convincing. Critically though, it posits that humans are basically asleep at the wheel during the gradual empowerment of the AI systems. Of course, Christiano posits that economic competition encourages this, but it still leaves a long period during which we can work on alignment or at least attempt to put the brakes on the automated systems.