3 ms·
Another risk is that, as the singularity seems closer and closer, people are going to throw up their hands, stop learning math and science figuring the machines
by tlb 7y ago
Another risk is that, as the singularity seems closer and closer, people are going to throw up their hands, stop learning math and science figuring the machines will do it all, and generally check out. If too many people do this too soon, sustainable recursive self-improvement won't happen and we'll be stuck with AI that doesn't quite work and nobody to fix it.
- conjectures 7y agoI don't think this is likely to happen. There is almost surely a class of interesting questions it takes AGI to answer but a math PhD to pose.
- cholmon 7y agoThe machine stops.
- gpderetta 7y agopeople still play chess even though machines are a class above.
- tlb 7y agoChess is fun, but will people enjoy fixing old security bugs? You could imagine an AI that's very good at adding features to itself, but just can't get security right because it doesn't think adversarially. Corporate software groups often have this same limitation.