3 ms·
Well a bunch of AI researchers are not gonna say AI is gonna kill us is 5 years. If they did, they would lose their jobs. If a self-improving AI could exist, i
by diskcat 11y ago
Well a bunch of AI researchers are not gonna say AI is gonna kill us is 5 years. If they did, they would lose their jobs.
If a self-improving AI could exist, it would be exponential in its self-improvement. So just like a supercritical nuclear reactor, it could conceivably go from undetectable level of activity to supreme intelligence within a very short span of time.
It would be like the nuclear engineer at chernobyl saying "is only 100MW comrade, is of not problem"
- GFK_of_xmaspast 11y ago> it would be exponential in its self-improvement Why do you think that is?
- diskcat 11y agobecause its ability to improve its intelligence is based on its intelligence i.e. exponential.
- umanwizard 11y agoThat assumes the marginal difficulty of further improvement stays constant as intelligence increases.
- GFK_of_xmaspast 11y agoIn addition to umanwizard's comment, you're assuming that there are no external constraints involved. There are plenty of non-exponential growth curves out there, like https://en.wikipedia.org/wiki/Logistic_function https://en.wikipedia.org/wiki/Logistic_function and https://en.wikipedia.org/wiki/Gompertz_function https://en.wikipedia.org/wiki/Gompertz_function , which you can imagine as being at least as probable as an exponential rate.
- argonaut 11y agoBy that same logic, Nick Bostrom has an incentive to exaggerate the risk because it sells him more books.