3 ms·
Prior to the development of nuclear weapons, there was well-grounded peer reviewed theory in the physics literature. There were predictions that could be made
by throwaway9274 3y ago
Prior to the development of nuclear weapons, there was well-grounded peer reviewed theory in the physics literature.
There were predictions that could be made by the theoretical frameworks, and experiments to confirm those predictions.
The literature well described the composition of atoms, the decay of radioactive elements, the concept of fission, and the conversion of mass to energy.
No such groundwork exists to substantiate the claim that runaway AI is possible.
LessWrong, the so-called Alignment Forum, and similar Internet groups propose thought experiments. They couch these thought experiments in the guise of inevitable outcomes.
The prophesies of the influencers originating there have no more relationship to the truth than the average enthusiast’s daydreams.
The content there fills a role similar to science fiction in the atomic age: fanciful speculation on the properties of imaginary systems.
Over time, the bad content is forgotten, and a handful of items survive and look prescient via selection bias.
- DirkH 3y agoDon't we likewise have predictions about FLOPs, compute, algorithms etc etc that are grounded in theoretical frameworks, and every year or so we make predictions and make experiments to confirm or deny AI capability predictions? The AI and neuroscience literature likewise describe the composition of intelligence and what it takes to build it. What would you need to see to believe we have enough evidence that runaway AI is possible, even if it is highly unlikely any time soon? (Worth adding here I don't know exactly what you mean by runaway AI other than simply AI humanity as a whole loses control over). I'm asking what it would take for you to go from 0% possible to 5% possible. Last time I checked science fiction authors in the atomic age making fanciful speculation on the properties of imaginary systems weren't responsible for actually building said systems. Though naturally, there will be exceptions. Nonetheless, I still have a hard time dismissing Geoffrey Hinton's concerns as fanciful sci-fi. Likewise, a nuclear world war actually wouldn't be as bad or even civilization-ending as fanciful sci-fi authors make it out to be. But dismissing its risks entirely because there are some nuclear-war forums where people come up with weird radiation thought experiments just seems preemptive to me.