2 ms·
Well, you asked for the steel man. It’s not easy for me to give one because I fundamentally disagree with most of the underlying assumptions. Superintelligence
by adastra22 3y ago
Well, you asked for the steel man. It’s not easy for me to give one because I fundamentally disagree with most of the underlying assumptions. Superintelligence is the standard work, and Tim does a decent summary of it.
Most of the AI existential risk concern comes down to whether a “singleton” will arise from AGI research, to use their terminology. A singleton is an entity that in a runaway increase of power ends up outwitting and outmaneuvering all other entities and becoming the single dominating, controlling force within the entire sphere of human existence.
There are so many reasons this is unlikely. It wouldn’t even be worth talking about if it weren’t for the sad fact so many otherwise smart people fall victim to this memeplex.
FYI timelines they are talking about are not so much 500 years as 5 months, or maybe 5 minutes.